KR20230121950A - Adaptive media streaming transmission method and apparatus for multi network - Google Patents

Abstract

A multi-network environment adaptive media streaming transmission method and apparatus are disclosed. Beauty according to an embodiment of the present invention
The Dear streaming transmission method includes the steps of encoding predetermined content into multiple layers to generate media data;
Separating the separated media data for each layer, and passing the media data separated for each layer through multiple networks.
and streaming and transmitting the solution to the media receiving device.

Description

Adaptive media streaming transmission method and apparatus for multi network environment adaptive media streaming transmission method and apparatus

[0001] The present invention relates to broadcast communication technology, and more particularly, to a media stream adaptively optimized for a network environment.

It is about Ming transmission technology.

[0002] Recently, various terminals that can connect to the Internet in a wired or wireless manner, such as smartphones, tablet PCs, smart TVs, etc., are available to users.

are being provided Multimedia services can be provided through these terminals, anytime, anywhere using the Internet.

It is evolving into a form of multimedia service in which all users select and consume the multimedia they want.

[0003] However, through the Internet where quality of service (QoS) is not guaranteed, a high-definition media streaming service without interruption is provided.

hard to provide In order to solve this problem, the hypertext transfer protocol (hypertext transfer protocol)

Hereinafter referred to as HTTP) based adaptive streaming service has been proposed. HTTP-based adaptive streaming service transports media

Media with different qualities according to various channel environments and device specifications targeting one content in the device

A sequence is created, and the media receiving device transmits media data to a designated channel according to its own terminal environment and transmission channel.

It is a technology that provides an optimized media streaming service as it is received through.

Representative companies of HTTP-based adaptive streaming technology include Microsoft, Apple, and Adobe. Microsoft in 2008

Smooth streaming technology was used for Internet live broadcasting of the Yijing Olympics, and Apple uses HTTP live streaming on iPhone and iPad.

Streaming technology was applied. These technologies correspond to proprietary non-standard HTTP-based adaptive streaming services.

In order to get out of the constraints of independent regulations, MPEG has developed Dynamic Adaptive HTTP Streaming (Dynamic Adaptive

Streaming over HTTP: hereinafter referred to as DASH) technology was standardized.

[0005] In providing Internet-based media streaming service, if DASH is used, regardless of channel bandwidth, minimum

Quality media data can be provided to users without interruption. For example, if the bandwidth of an IP network is small,

If the bandwidth is large, the video stream with the minimum quality is transmitted. Conversely, the video stream with the maximum quality is transmitted.

send Accordingly, video data encoded with an appropriate bit rate according to the transmission band environment of the IP network is transmitted.

Therefore, the user can watch the video without interruption.

[0006] On the other hand, since DASH provides flexible quality of service according to channel conditions in a single IP network, users always

As a result, media data of the same quality cannot be received. In other words, DASH is a high-definition image depending on the bandwidth situation of the IP network.

Since low-quality images are transmitted to the user, only high-quality images cannot be transmitted all the time. example

For example, if the bandwidth of the user's IP network channel is narrow, high-quality video cannot be streamed.

[0007] According to an embodiment, in order to always provide a high-quality media streaming service to users, multiple network environments

An adaptive media streaming transmission method and device are proposed.

[0008] A media streaming transmission method of a media transmission device according to an embodiment encodes predetermined content into multiple layers.

The step of generating media data, the step of separating the created media data for each layer, and the step of classifying the created media data for each layer.

and streaming the processed media data to a media receiving device through a plurality of networks.

[0009] A media transmission device according to another embodiment generates media data by encoding predetermined content into multiple layers.

An encoding unit, a layer control unit for allocating layers for encoding in the encoding unit according to the number of available networks, and an encoding unit

A layer separation unit that separates the media data generated in each layer by layer, and the media separated by layer in the layer separation unit

and a media transmission unit for streaming data to a media reception device through multiple networks.

[0010] A media streaming receiving method of a media receiving device according to another embodiment includes recognizing predetermined content in a plurality of layers.

Receiving media data generated by coding from a media transmission device through multiple networks;

and decoding the dear data.

[0011] A media receiving device according to another embodiment includes media data generated by encoding predetermined content into multiple layers.

A media receiver for receiving data from a media transmission device through multiple networks;

and a decoding unit that decodes the data.

[0012] According to an embodiment, the bandwidth of a single network channel in a streaming technology adaptive to an existing network environment

Overcome the limitations of providing low-quality media services due to limitations, and optimize adaptively for multiple network environments.

High-quality media streaming service can always be provided through optimized media streaming transmission.

[0013] Specifically, the existing advanced video coding (Advanced Video Coding: AVC) based environment adaptive streaming technology is

When providing a streaming service, only a single IP network should be used. However, according to the present invention, the scalable ratio

By encoding the content into multiple layers using Scalable Video Coding (SVC) and separating the layers,

It is possible to stream and transmit high-quality media data through multiple IP networks.

1 is a configuration diagram of a multi-network environment adaptive media streaming transmission system according to an embodiment of the present invention;
2 is a detailed configuration diagram of a media streaming transmission system according to an embodiment of the present invention;
3 is a configuration diagram of a media transmission device according to an embodiment of the present invention;
4 is a configuration diagram of a media receiving device according to an embodiment of the present invention;
5 is a flowchart illustrating a media streaming transmission method of a media transmission device according to an embodiment of the present invention;
6 is a flowchart illustrating a media streaming receiving method of a media receiving apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Related to the description of the present invention

If it is determined that a detailed description of a known function or configuration may unnecessarily obscure the gist of the present invention,

A detailed description thereof will be omitted. In addition, the terms to be described later are terms defined in consideration of functions in the present invention.

This may vary depending on the intention or custom of the user or operator. Therefore, the definition is throughout this specification.

It should be based on content.

1 is a diagram of a multi-network environment adaptive media streaming transmission system 1 according to an embodiment of the present invention.

It is a composition diagram.

Referring to Figure 1, the media streaming transmission system 1 is composed of a media transmission device 10 and a plurality of IP networks

It includes a network 12 and a media receiving device 14.

[0019] The media transmission device 10 transmits media data, for example, video data to the media reception device 14 by streaming.

However, streaming is transmitted through the network 12 composed of a plurality of IP networks. The media receiving device 14 is a network

The media data is streamed and received from the media transmission device 10 by accessing the network 12.

[0020] The network 12 is composed of a plurality of IP networks, a main IP network and a sub IP network as shown in FIG.

includes work There may be multiple sub IP networks. An IP network is, for example, a wired network such as 3G or 4G.

It may be a wireless network such as a network or wifi, the Internet, an IPTV network, and the like.

[0021] The media transmission device 10 receives a media streaming service adaptively optimized for a plurality of IP network environments.

provided to the device 14. Accordingly, the media receiving device 14 determines the status of a plurality of IP networks, for example, each IP network.

Regardless of the transmission bandwidth of the network, high-quality streaming service can always be provided. Media transmission device for this purpose

(10) encodes the content in multiple layers using Scalable Video Coding (hereinafter referred to as SVC) technology, and then streams the content to the media receiver 14 through multiple IP network channels. .

[0022] SVC, an extended encoding technology of H.264, has scalability based on hierarchical encoding attempted in MPEG-2, MPEG-4, etc.

In order to solve the problems of low compression efficiency, inability to support complex scalability, and high implementation complexity,

It is a new scalable encoding technique developed by SVC combines multiple video layers into one bitstream

encode with The layers of SVC consist of one base layer and an enhancement system that can be successively stacked on top of the base layer.

It consists of an enhancement layer. Each enhancement layer has the maximum bit rate given to each based on the information of the lower layer.

(bit-rate), frame rate, and resolution can be expressed. In SVC, successively many enhancement layers are

As they are stacked, various bit rates, screen rates, and resolutions can be supported. Therefore, in a heterogeneous network environment

bandwidth diversity problem, diversity problem of receiving terminal performance and resolution, and various preferences of content consumers

multimedia content service in UMA (Universal Multimedia Access) environment that can solve problems

It is a suitable encoding technique.

[0023] According to the present invention, the media transmission apparatus 10 encodes and encodes predetermined content as many as the number of layers using SVC.

Media data constituting one bitstream consisting of a number of layers is generated. For example, if the video layer is

If it is composed of one basic layer and two enhancement layers, the media transmission device 10 has a total of three layers.

Media data constituting the SVC bitstream is generated.

2 is a detailed configuration diagram of a media streaming transmission system 1 according to an embodiment of the present invention.

[0026] Referring to FIG. 2, the media streaming transmission system 1 transmits multiple network environment adaptive streaming based on SVC.

have a structure Specifically, the media transmission device 10 recognizes the content based on multiple layers through the encoding unit 320.

Coding to generate media data. Encoded media data is composed of one SVC bitstream, in FIG.

As shown, if the available video layer consists of one base layer and two enhancement layers, the encoding unit

Step 320 generates media data constituting one SVC bitstream consisting of a total of three layers.

[0027] The layer separation unit 330 separates the media data encoded in the encoding unit 320 for each layer and converts them into a transmission format.

brighten up For example, as shown in FIG. 2, the layer separation unit 330 separates media data into one base layer and two directions.

separated into upper layers. Subsequently, the media transmission device 10 transfers the media data converted to the transmission format to each designated folder.

After storing in 370, it is streamed to the media receiving device 14 through the network 12 composed of multiple IP networks.

send.

[0028] On the other hand, the media receiving device 14 transmits the media data transmitted from the media transmitting device 10 through the network 12 to the media

It is received through the receiver 430. Then, the received media data is decoded through the decoding unit 450 .

[0029] Hereinafter, detailed configurations of the media transmission device 10 and the media reception device 14 will be described with reference to FIGS. 3 and 4, which will be described later.

And, with reference to FIGS. 5 and 6, the process of each device 10 and 12 will be described later.

3 is a configuration diagram of a media transmission device 10 according to an embodiment of the present invention.

[0032] Referring to FIGS. 1 and 3, the media transmission device 10 includes a layer control unit 310, an encoding unit 320, a layer separator 330, a segment

It includes a site generator 340, a metadata generator 350, and a media transmission unit 360.

[0033] The layer control unit 310 allocates layers for encoding by the encoding unit 320 according to the number of available networks. In other words, hierarchical control

The unit 310 allocates SVC encoding layers to the encoding unit 320 according to the number of IP networks available to the provider. in other words

If the provider can use three heterogeneous IP network channels, the encoding unit 320 converts the content into a basic layer and two enhancements.

You can encode in layers.

[0034] The encoding unit 320 encodes predetermined content into multiple layers to generate media data. At this time, the encoding unit 32

0) encodes the content according to the number of layers allocated by the layer controller 310. Encoded binary data is

It consists of a bit stream. The encoding unit 320 may be located in a video coding layer (VCL).

[0035] The layer separator 330 separates the media data generated by the encoding unit 320 for each layer. That is, the encoding unit 320

The bitstream encoded by is separated for each layer by the layer separation unit 330 . Media data per separated layer is transmitted

converted to format. The layer separation unit 330 may be located in a Network Abstraction Layer (NAL).

[0036] The segment generator 340 separates the media data for each layer through the layer separator 330 and then converts the media data into a transport format.

is divided into a plurality of segment data. At this time, the segment generator 340 converts the media data for each layer in time.

It can be divided into units (e.g. units of seconds). Divided segments can be stored on designated servers.

[0037] The metadata generating unit 350 targets the media data divided into segments through the segment generating unit 340 for each step.

A metadata file is created for each layer, and the generated metadata file is transmitted to the media receiving device 14 through multiple networks.

send. The metadata file generated by the metadata generator 350 may be determined according to the number of layers. for example

When the number of layers encoded by the encoding unit 320 is three, the metadata file generated by the metadata generation unit 350 is also

It becomes three. In other words, if the number of layers is 3, this means that the provider has at most 3 heterogeneous IP network channels available.

all.

[0038] In the metadata file, segment information and codec information are defined, and an access IP address called URL is displayed. segment

Information and codec information may be obtained from the segment generator 340, and URL information may be obtained from the layer control unit 310.

there is. As an example of URL information, when the number of layers is three, the metadata generating unit 350 displays the URL of the main IP network.

Create one metadata file and two metadata files showing the URLs of the other sub-IP networks. meta

All metadata generated by the data generator 350 is stored in the media when the media receiving device 14 accesses multiple networks.

All are transmitted to the receiving device 14. The media receiving device 14 requires metadata generated based on the main IP network.

and analyze metadata created based on other sub-IP networks depending on the situation.

[0039] The media transmission unit 360 divides the segmented segments through the segment generation unit 340 after being separated for each layer through the layer separation unit 330.

messages are streamed to the media receiving device 14 through multiple networks. Segments are base layer segments

and an enhancement layer segment.

[0040] According to an embodiment, the media transmission unit 360 preferentially transmits the segments through the main IP network, and

On request, the remaining segments are transmitted through at least one sub-IP network. For example, the media transport unit

360 preferentially transmits the base layer segment through the main IP network. At this time, the main IP network

If the network condition of is unstable, for example, the channel bandwidth of the main IP network can be segmented in all layers.

If unavailable, first move to a layer that can be accommodated by the main IP network, including the basic layer segment.

The remaining segments, including the enhancement layer segments, are transmitted through the main IP network.

can be transmitted through a sub IP network.

4 is a configuration diagram of a media receiving device 14 according to an embodiment of the present invention.

1 and 4, the media receiving device 14 includes a channel analyzer 410, a streaming controller 420, and a media receiver.

430, a synchronization unit 440 and a decoding unit 450.

[0044] The media receiving device 14, which is a client, connects to the network 12 including a plurality of IP networks and transmits the media.

Streaming media data is received from (10). A number of IP networks include, for example, the Internet, IPTV networks,

It includes wireless networks such as wifi, wired networks such as 3G and 4G, and the like. The media receiving device 14 based on the SVC

Media data optimized for the network environment or client environment by dynamically adapting to multiple IP network environments

receive streaming transmission. Hereinafter, the detailed configuration of the media receiving device 14 having the above characteristics will be described in detail.

[0045] The channel analyzer 410 analyzes the network state for each of a plurality of IP networks to determine the network state for available networks.

Information is provided to the streaming controller 420 . At this time, the channel analysis unit 410 checks available IP networks and performs

Intermittently monitors the network status such as the bandwidth of the IP network and transmits the network status information to the streaming control unit 420

can be provided to

[0046] When the media transmission device 10 encodes the content into multiple layers and then creates a metadata file for each layer, the stream

The ming controller 420 receives a metadata file for each layer from the media transmission device 10 .

[0047] Next, the streaming controller 420 analyzes the metadata file for each layer received from the media transmission device 10 and the channel.

At least one network to be accessed by the media receiving device 14 is determined using the network status information received from 410.

decide According to an embodiment, the streaming control unit 420 checks the number of metadata files and the media receiving device 14

Determine the number of accessible IP networks. Specifically, the streaming control unit 420 checks the HTTP-URL information of the segment encoded in the media transmission device 10 from the metadata file. And, received from the channel analyzer 410

Depending on the IP network status information, whether to receive streaming service using multiple IP networks or a single IP network

Determines whether or not to receive a streaming service using a work.

[0048] The media receiver 430 receives a plurality of encoded segments from the media transmission device 10. Segment is the base layer

It includes segments and enhancement layer segments. The media receiving unit 430 opens all accessible HTTPs and streams them.

As commanded by the controller 420, segments are received from the media transmission device 10 through the assigned HTTP address. Day

According to an embodiment, when the channel bandwidth of the main IP network cannot accommodate all layers, the media receiving unit 430

Segments up to the layer that the main IP network can accommodate are received, and segments of the remaining layers that are not received are received.

are received through other sub-IP networks.

[0049] Synchronization unit 440 is the media receiving unit 430 through the media transmission device 10 received from the segments of the media data

synchronize each other. At this time, the synchronization unit 440 divides the video segment groups and channels analyzed by the streaming control unit 420.

It is determined how many layers to synchronize using the network state information provided from the analyzer 410, and

can synchronize the segments of the layer. According to an embodiment, the synchronizing unit 440 determines the IP network status and current usage.

Check the number of IP networks being used to check the layer of the received segment. At this time, the number and transmission of IP networks

When the number of layers that have been assigned is the same, synchronization may be performed using a timestamp value.

[0050] The decoding unit 450 receives the synchronized segments from the synchronization unit 440 and converts them from a transport format to a video format.

convert Then, the segments converted to the video format are decoded.

5 is a flowchart illustrating a media streaming transmission method of the media transmission device 10 according to an embodiment of the present invention.

[0052] Referring to FIGS. 2 and 5, the media transmission device 10 encodes predetermined content into multiple layers to obtain media data.

It is created (5000), and the created media data is separated for each layer (5010). According to one embodiment, media data production

In the formation step 5000, the media transmission device 10 encodes a predetermined content as many as the number of layers and converts the content into the encoded number of layers.

Media data constituting one bitstream formed is generated. And, in the layer-by-layer separation step 5010,

Media data constituting my bitstream is separated into a basic layer and an enhancement layer.

[0053] Subsequently, the media transmission device 10 transfers media data separated for each layer to a media reception device through a plurality of networks.

Streaming is transmitted to (14) (5020).

[0054] According to an embodiment, in the streaming transmission step 5020, the media transmission device 10 separates media data for each layer.

Convert each to a transport format, divide the media data converted to the transport format into segments, and divide into segments.

The divided media data is transmitted to the media receiving device 14 through multiple networks. At this time, the media transmission device 10

stores the segments in each designated folder, and connects the segments stored in each designated folder to the main IP network.

It can be transmitted preferentially through at least one sub-IP network if necessary.

[0055] According to an embodiment, in the streaming transmission step 5020, the media transmission device 10 separates media data for each layer.

For each layer, a metadata file is created for each layer, and the metadata file created for each layer is distributed over multiple networks.

transmit through the At this time, the metadata file is at least one of partition information, codec information, and access IP address information.

includes

6 is a flowchart illustrating a media streaming receiving method of the media receiving device 14 according to an embodiment of the present invention.

[0058] Referring to FIGS. 2 and 6, the media receiving device 14 encodes predetermined content into multiple layers and generates media.

Data is received from the media transmission device 10 through the network 12 composed of a plurality of IP networks (6000). picture

and decodes the received media data (6010).

[0059] According to an embodiment, the media receiving device 14 transmits a metadata file generated for each layer after encoding the media data.

It is received from the Dear transmission device 10. Then, the media receiving device 14 uses the received metadata file for each layer.

Decide which network to connect to. In this case, the media receiving device 14 in the media data receiving step 6000, the determined four

It connects to the network and receives media data from the media transmission device 10.

[0060] The media data received from the media transmission device 10 is composed of segments after being encoded in a plurality of layers.

all. According to an embodiment, in the media data receiving step 6000, the media receiving device 14 first receives segments up to a layer that the main IP network can accommodate through the main IP network. And, the main IP

If all segments have not been received through the network, at least one

It is received through the sub-IP network.

[0061] According to one embodiment, in the decoding step 6010, the media receiving device 14 is a media transmitting device through a plurality of networks.

Synchronize the segments of media data received from (10) and decode the synchronized segments. At this time, layer

The number of segments to be synchronized is determined using information and network state information, and the number of segments to be synchronized is determined.

can synchronize them.

[0062] So far, the present invention has been looked at focusing on the embodiments. Common knowledge in the art to which the present invention belongs

Those who have the present invention can be implemented in a modified form to the extent that does not depart from the essential characteristics of the present invention

You will understand. Therefore, the disclosed embodiments should be considered from a descriptive point of view rather than a limiting point of view.

do. The scope of the present invention is shown in the claims rather than the foregoing description, and all within the scope equivalent thereto.

Any differences should be construed as being included in the present invention.

[0063] 1: media streaming transmission system 10: media transmission device
12: network 14: media receiving device
310: layer control unit 320: encoding unit
330: layer separation unit 340: segment generation unit
350: metadata generation unit 360: media transmission unit
410: channel analysis unit 420: streaming control unit
430: media receiving unit 440: synchronization unit
450: decoding unit

Claims (15)

In the media streaming transmission method of the media transmission device,
generating media data by encoding predetermined content into multiple layers;
Separating the generated media data for each layer; and
It includes the step of streaming the media data separated for each layer to a media receiving device through a plurality of networks.
including,
In the step of transmitting the streaming,
converting the separated media data for each layer into a transport format;
dividing the media data converted into a transport format into segments; and
transmitting the media data divided into segments through the plurality of networks;
The step of transmitting the media data divided into the segments through the plurality of networks,
Saving each segment to a designated folder; and
Segments stored in each of the designated folders are preferentially transmitted through the main IP network, and the remaining segments
A media stream comprising the step of transmitting the streams through at least one sub-IP network.
Reaming transmission method. 2. The method of claim 1, wherein generating the media data comprises:
A bit stream consisting of the encoded number of layers by encoding a predetermined content as many as the number of layers.
create deer data,
The step of separating each layer is
Media characterized in that media data constituting one bitstream is separated into a basic layer and an enhancement layer.
Streaming transmission method. The method of claim 1, wherein the streaming transmission comprises:
generating a metadata file for each layer, targeting media data separated for each layer; and
Transmitting the metadata file generated for each layer through the plurality of networks;
Media streaming transmission method comprising a. According to claim 3,
The metadata file includes at least one of partition information, codec information, and access IP address information.
A media streaming transmission method. an encoding unit generating media data by encoding predetermined content into a plurality of layers;
a layer controller allocating a layer for encoding by the encoding unit according to the number of available networks;
a layer separator separating the media data generated by the encoding unit for each layer;
a segment generation unit dividing the media data separated by each layer in the layer separation unit into segments;
a storage unit to store each of the segments of the media data divided by the segment generation unit; and
The segments stored in the storage unit are preferentially transmitted through the main IP network, and the remaining segments are loaded.
a media transmitter for transmitting through at least one sub-IP network;
A media transmission device comprising a. According to claim 5,
The segment generator generates metadata files for each layer targeting the media data divided into segments.
a metadata generation unit for generating and transmitting generated metadata files through the plurality of networks;
A media transmission device characterized in that it further comprises. In the media streaming receiving method of the media receiving device,
Media transmission device through multiple networks for media data generated by encoding predetermined content into multiple layers
receiving from; and
decoding the received media data;
The media data received from the media transmission device is composed of segments after being encoded in multiple layers.
go,
Receiving from the media transmission device through the plurality of networks,
First of all, segments up to the layer that the main IP network can accommodate are transmitted through the main IP network.
and, if all segments are not received through the main IP network, the remaining layers not received
Media streaming reception, characterized in that for receiving a segment of through at least one sub-IP network
method. According to claim 7,
Receiving from the media transmission device a metadata file generated for each layer after media data encoding;
and
At least one network to be accessed by the media receiving device is determined using the received metadata file for each layer.
determining step; Including more,
Receiving the media data from a media transmission device through multiple networks includes:
Receiving media data from the media transmission device by accessing the determined at least one network.
Media streaming receiving method characterized by. 9. The method of claim 8, wherein the step of determining at least one network to which the media receiving device is to access comprises:
By checking the number of metadata files received from the media transmission device, the media receiving device can access
determining the number of networks; and
Determining at least one network to be accessed from among accessible networks using network status information for each network identified through access IP address information included in the metadata file;
Media streaming receiving method comprising a. 8. The method of claim 7, wherein decoding the received media data comprises:
Synchronizing segments of media data received from the media transmission device through the plurality of networks
total; and
decoding the synchronized segments;
Media streaming receiving method comprising a. 11. The method of claim 10, wherein synchronizing segments of the received media data comprises:
The number of segments to be synchronized is determined using layer information and network state information, and the number of synchronized segments is determined.
A method for receiving media streaming, characterized in that synchronizing the segments. Media transmission device through multiple networks for media data generated by encoding predetermined content into multiple layers
a media receiving unit receiving from; and
a decoding unit for decoding the media data received by the media receiving unit;
The media data received from the media transmission device is composed of segments after being encoded in multiple layers.
go,
The media receiver,
First of all, segments up to the layer that the main IP network can accommodate are transmitted through the main IP network.
and, if all segments are not received through the main IP network, the remaining layers not received
A media receiving device characterized in that for receiving a segment of at least one sub-IP network. According to claim 12,
After media data encoding, a metadata file generated for each layer is received from the media transmission device, and
At least one network to be accessed by the media receiving device is determined using each metadata file and network state information.
a streaming control unit that determines; Including more,
The media receiver,
A media file is transmitted from the media transmission device through at least one network determined by the streaming control unit.
Media streaming receiving device, characterized in that for receiving. According to claim 13,
Analyzing the state of each network and providing network state information on available networks to the streaming control unit
a channel analysis unit;
Media receiving device characterized in that it further comprises. According to claim 12,
a synchronizing unit that synchronizes segments of media data received from the media transmission device through the plurality of networks;
Media streaming receiving device characterized in that it further comprises.