KR102291293B1 - Transmission device, comunication system, transmission method, and non-transitory computer readable recording medium - Google Patents

Abstract

A transmission device according to an embodiment includes one or more processors. The processors divide multiple types of transmission data to be transmitted into first data and second data. The processors transmit the first data to a server device configured to distribute the transmission data to a reception device. The processors store the second data in storage. The processors receive, from the reception device or the server device, a transmission request of the second data. The processors transmit the second data to the server device according to the transmission request.

Description

Transmission device, communication system, transmission method, and non-transitory computer-readable recording medium

This application is based on and claims the benefit of priority from Japanese Patent Application No. 2018-207604, filed on November 2, 2018; The entire contents of which are incorporated herein by reference.

Embodiments described herein relate generally to a transmitting device, a communication system, a transmitting method, and a computer program product.

Adaptive streaming such as HTTP Live Streaming (HLS) and Moving Picture Experts Group-Dynamic Adaptive Streaming over HTTP (MPEG-DASH), for example, distributes and monitors images (moving data) captured by a camera It can be applied to a system (image distribution system or image monitoring system) for

However, in the prior art, there is a risk of an increase in the load on the network line through which data (such as an image) is transmitted and an increase in the processing load on the system. For example, as the number of connected cameras increases, a load on network lines for transmitting images from the cameras and a processing load on a server executing recording and distribution of images potentially increase.

It is an object of the embodiments to provide a transmission device, a communication system, a transmission method, and a computer program product capable of reducing line load and processing load without degrading quality.

According to one embodiment, a transmitting device according to the embodiment comprises one or more processors. The processors divide the plurality of transmission data to be transmitted into first data and second data. The processors transmit the first data to a server device configured to distribute the transmit data to a receiving device. The processors store the second data in storage. The processors receive, from the receiving device or the server device, a request to transmit the second data. The processors transmit the second data to the server device according to the transmission request.

According to the above-described transmitting device, the line load and the processing load can be reduced without degrading the quality.

1 is a block diagram of an image distribution system according to the present embodiment;
Fig. 2 is a diagram showing an outline of distribution processing performed by the image distribution system according to the present embodiment;
3 is a functional block diagram of devices of the image distribution system according to the present embodiment;
4 is a diagram illustrating an exemplary data structure of a distribution list;
5 is a diagram illustrating an exemplary data structure of decision information;
6 is a flowchart of transmission processing in the present embodiment;
7 is a flowchart of stored data transmission processing;
Fig. 8 is a flowchart of list generation processing in the present embodiment;
Fig. 9 is a flowchart of determination information generation processing in the present embodiment;
Fig. 10 is a sequence diagram of content distribution processing in the present embodiment;
Fig. 11 is a diagram for explaining an example of making a judgment on a distribution list based on judgment information;
12 is a diagram illustrating exemplary updated decision information;
Fig. 13 is a diagram for explaining an example of making a judgment on a distribution list based on judgment information;
14 is a diagram showing an exemplary configuration of an image distribution system according to a modification;
Fig. 15 is a diagram showing an outline of distribution processing performed by an image distribution system according to a modification;
16 is a hardware configuration diagram of a transmission device according to the present embodiment.

Preferred embodiments of a transmitting device will be described in detail below in conjunction with the accompanying drawings.

In a video distribution system using the conventional adaptive streaming, a network line having a high transmission speed needs to be prepared to reliably record a high-quality video stream (exemplary transmission data) necessary for distribution of high-quality live video. .

In a large-scale image distribution system in which a plurality of cameras are connected, a large amount of camera images are continuously transmitted. Therefore, in order to reduce line load and cost, it is preferable to use a network line with a low transmission speed between each camera and a server device for recording an image. However, live video quality is poor on these lines with low transmission rates, which potentially creates problems for monitoring tasks.

Accordingly, the present embodiment achieves an image distribution system capable of reducing line load and processing load without degrading quality.

Hereinafter, exemplary transmission data to which the present embodiment can be applied will be described. The transmission data includes time series data whose temporal order is determined, such as moving image data or sensor data.

The moving picture data is captured by an imaging device such as a camera or frame capture. For example, moving image data is acquired in real time from the imaging device and used as content to be distributed. The moving image data temporarily stored in the storage medium after being photographed can be used as content to be distributed.

The sensor data represents a value sensed by a sensor (sensing device) and includes, for example, information of a time at which the data is sensed (sampled). The sensor may be any device. The sensor periodically or aperiodically senses, for example, a microphone configured to acquire voice, a global positioning system (GPS) device configured to acquire location information, or the temperature, speed, pressure, etc. of the surrounding environment or electronic device to be sensed. Thus, it may be a sensor that outputs the sensed value as sensor data.

An example in which the transmission data is moving picture data will be mainly described below.

1 is a block diagram showing an exemplary configuration of an image distribution system according to the present embodiment. As shown in FIG. 1 , the image distribution system includes a transmission device 100 (an exemplary transmitting device), a server 200 (an exemplary server device), and a client 300 (an exemplary receiving device). The transmission device 100 and the server 200 are connected to each other through a network 401 . The server 200 and the client 300 are connected to each other through a network 402 .

Networks 401 and 402 may be any networks, such as the Internet. For example, networks 401 and 402 may each be a wired or wireless network. Networks 401 and 402 may be configured as one integrated network.

The configuration of the communication system shown in FIG. 1 is only an example, and the present embodiment is not limited thereto. For example, a plurality of transmission devices 100 , a plurality of servers 200 , and a plurality of clients 300 may be provided. Each of the transmitting device 100 , the server 200 , and the client 300 may be physically configured as one device or may be physically configured as a plurality of devices. For example, the server 200 may be built in a cloud environment.

Fig. 2 is a diagram showing an outline of distribution processing performed by the image distribution system according to the present embodiment. The transmitting device 100 receives an image and encodes the received image. The transmission device 100 encodes an image by compression according to a standard such as H.264. The encoded video includes, for example, I pictures and P pictures. An I picture is picture data encoded by intra-frame prediction, and includes the entire screen range that can be reproduced alone. A P picture is picture data encoded by inter-frame prediction based on an I picture. A P picture cannot be reproduced alone, but can be reproduced in combination with an I picture.

The transmission device 100 divides (fragments) the encoded image for each picture. Next, the transmission device 100 transmits the divided picture part (eg, I pictures) to the server 200, and stores the remaining picture part (eg, P pictures) in storage. The transmission device 100 may also store the pictures transmitted to the server 200 in storage for playback (local playback), backup, and the like.

Typically, the server 200 distributes only the I pictures transmitted from the transmitting device 100 to the client 300 . The client 300 displays the distributed image using, for example, a viewer application. The viewer application is, for example, included in the browser and configured to display images for browsing. I pictures are transmitted at regular intervals (e.g., several pictures for 1 second or one picture for several seconds), and thus images (still-motion animation or frame-by-frame) in which still pictures (I pictures) are updated at regular intervals A playback (intermittent) moving image) is displayed on the client 300 . The viewer application may be achieved as an application using Media Source Extensions (MSE) as an application programming interface for, for example, Hyper Text Markup Language (HTML) 5 . This enables streaming playback using HTTP downloading.

When display of the P picture is requested by a user operating the client 300 or the like, the transmission device 100 reads the stored P picture and transmits the read P picture to the server 200 . The server 200 distributes the transmitted P picture to the client 300 . The server 200 can directly distribute the transmitted P picture, or a P picture that is applied only to processing (eg, change of transmission packet format) other than conversion processing (eg, recompression) for image data. can be distributed. The viewer application of the client 300 synthesizes and displays the already received I picture and the later received P picture. Accordingly, the client 300 may display a smoother image. The forwarded data (eg, I pictures) can be stored in storage within the client 300 and used for compositing with, for example, P pictures that are received later. That is, the forwarded data does not need to be transmitted again from the transmitting device 100 and the server 200 . This method can achieve a reduction in the amount of transmission data compared to a method in which the entire image data including P pictures is transmitted again.

As described above, for example, without providing conversion processing such as recompression for image data, the server 200 processes the image data transmitted from the transmission device 100 directly or after processing only the transmission packet format. , distributed to the client 300 . For example, the server 200 does not need to execute a process of converting an image with reduced quality for distribution at a low transmission rate. Accordingly, an increase in the processing load on the server 200 and a decrease in image quality can be avoided. Also, the amount of communication can be reduced by distributing only I pictures during normal operation. In this way, the line load and processing load can be reduced without quality degradation.

An I picture is partial data included in a video (moving picture data), but can be treated as a still picture. For example, a browser (viewer application) operating as the client 300 may, in some cases, display the I-picture included in the moving image data as still image data. Therefore, for example, I-pictures can be displayed as still image data without performing compression processing to still images such as JPEG (Joint Photographic Experts Group) images. Since the I picture has a small data size (because the compression efficiency is high) compared to that of a still picture of the JPEG format, the line load and processing load can be further reduced. When the distributed image is used for image recognition, the I-picture can be used as input data for image recognition without converting the image into a still image.

Hereinafter, the configurations of the devices of the image distribution system according to the present embodiment will be described in detail. 3 is a block diagram illustrating exemplary functional configurations of devices of the image distribution system according to the present embodiment.

3 , the transmission device 100 includes an imaging unit 101 , a storage 121 , an encoding unit 111 , a dividing unit 112 , a data transmission unit 113 , and a storage control unit 114 . and a request receiving unit 115 .

The imaging unit 101 captures and outputs an image (moving image data). The imaging unit 101 is achieved, for example, by an image sensor such as a charge coupled device (CCD) or a CMOS image sensor (CIS), or a frame buffer capture such as a frame memory, frame grabber, or screen capture. can be

The storage 121 stores various types of data used in various types of processing performed by the transmission device 100 . For example, the storage 121 stores an image captured by the imaging unit 101 .

The encoding unit 111 encodes the image input from the imaging unit 101 . Encoding by the encoding unit 111 may be performed by any method, for example, an encoding method according to a standard such as H.264. For example, the encoding unit 111 encodes an image by compression, and outputs an image including I and P pictures. Pictures of the encoded image correspond to a plurality of transmission data to be transmitted.

The dividing unit 112 divides the encoded image into data to be transmitted to the server 200 (first data) and data to be stored in the storage 121 without transmission (second data). For example, the dividing unit 112 encodes each I picture to be data to be transmitted to the server 200 (fragment data) and each P picture to be data to be stored in the storage 121 (fragment data). Divide the image into pictures. The dividing unit 112 may divide the image so that each divided picture is provided as one file (fragment file). The dividing unit 112 may divide the plurality of pictures into data to be stored in the storage 121 so as to be included in one file.

The data division by the dividing unit 112 is not limited to this method and may be performed by any method. For example, the dividing unit 112 may divide the encoded image into I pictures selected at regular intervals from a plurality of I pictures, and other pictures (remaining I pictures and P pictures). Alternatively, for example, the dividing unit 112 may divide the encoded image into I pictures, P pictures selected at regular intervals from among a plurality of P pictures, and other pictures (remaining P pictures). have. Alternatively, for example, the dividing unit 112 divides the encoded image into I pictures selected at regular intervals from among a plurality of I pictures, P pictures selected at regular intervals from among a plurality of P pictures, and other pictures (remaining I pictures and P pictures).

Data to be transmitted to the server 200 may be expressed in a format directly distributable by the server 200 . For example, the segmentation unit 112 may convert the segmented data into a format according to a standard such as fragmented MP4 (fMP4).

Data to be transmitted to the server 200 may be expressed in a format with added data (metadata) that may be used for conversion into a format in which the data is distributed by the server 200 . For example, the segmentation unit 112 may add metadata including information necessary for conversion by which the server 200 may perform conversion into a format conforming to a standard such as fMP4, to the segmented data. Information necessary for conversion includes, for example, information (eg, IP address and port number) for identifying the transmitting device 100, time (eg, day, hour, minute, second), and time It includes the position of the image in the . (for example, information indicating the order of the image from the head).

The dividing unit 112 may change the size or encoding amount of data to be transmitted to the server 200 according to the band of the network 401 . For example, the dividing unit 112 may generate data corresponding to a size that can be transmitted without delay or the like within the allocated band of the network 401, or data having an encoding bit rate within the band of the network 401, It can be divided as data to be transmitted to the server 200 . For example, when the band of the network 401 is large, the dividing unit 112 may divide all I pictures and some P pictures as data to be transmitted to the server 200, and as the band decreases, the server As data to be transmitted to 200 , only I pictures or some of the I pictures (eg, by thinning the I pictures at regular intervals) may be divided. When some P pictures are to be distributed, encoding can be performed, for example, by the method disclosed in Japanese Patent No. 6239472.

The segmentation unit 112 dynamically switches the segmentation method as described above according to the bandwidth of the network 401 or a request from the client 300 or the server 200 to thereby encode the size or encoding of data to be transmitted to the server 200 . quantity can be changed.

When data encoded by the encoding unit 111 is simply divided, the size of each divided data is potentially unequal. For example, when the encoding unit 111 has a function of adjusting the encoding amount of each picture according to the band of the network 401, the sizes of I and P pictures are different from each other. Thus, for example, when an I picture among pictures encoded in this way is divided as data to be transmitted to the server 200, the sizes of the divided I pictures are potentially different from each other.

Accordingly, the encoding unit 111 adjusts the encoding amount according to the band of the network 401 so as to obtain a size or encoding bit rate at which the divided data can be transmitted without delay or the like within the allocated band of the network 401 . can have it For example, when the dividing unit 112 divides an image such that only I pictures are transmitted to the server 200 , the encoding unit 111 is configured such that the I pictures are transmitted without delay or the like within the band of the network 401 . The image may be encoded to have a certain size or to have an encoding bit rate within the band of the network 401 .

The data transmitting unit 113 transmits data to an external device such as the server 200 . For example, the data transmission unit 113 transmits to the server 200 data (first data) to be transmitted to the server 200 among the data divided by the division unit 112 . When a request to transmit data stored in the storage 121 is received by the request receiving unit 115 (to be described later), the data transmitting unit 113 transmits the requested data to the server 200 .

The storage control unit 114 controls the processing of storage to the storage 121 . For example, the storage control unit 114 stores data not transmitted to the server 200 among the data divided by the division unit 112 in the storage 121 . The storage control unit 114 may store the data (first data) transmitted to the server 200 in the storage 121 . In this case, the storage control unit 114 may manage the stored data by using metadata indicating that the transmitted data (first data) has been transmitted or that the second data has not been transmitted. The storage control unit 114 may delete data stored in the storage 121 according to a predetermined condition. For example, the storage control unit 114 may delete stored data for a certain duration.

The request receiving unit 115 receives a transmission request of data stored in the storage 121 from the server 200 . When a request is transmitted from the client 300 through a control server other than the server 200 , the request receiving unit 115 may receive a transmission request of data stored in the storage 121 from this control server.

The above-described components (encoding unit 111 , partitioning unit 112 , data transmitting unit 113 , storage control unit 114 , and request receiving unit 115 ) are each performed by, for example, one or more processors. is achieved For example, each component may be accomplished by a computer program executed by a processor such as a central processing unit (CPU), ie, by software. A component may be accomplished by a processor, such as a dedicated integrated circuit (IC), ie, by hardware. A component may be achieved by software and hardware. When multiple processors are used, each processor may achieve one of the components, or may achieve two or more of the components.

Functions of the transmission device 100 may be distributed to a plurality of devices that are physically or logically different from each other. For example, the functions may include a device including an imaging unit 101 and an encoding unit 111 (image input and encoder in FIG. 2 ), and a device including the remaining components (fragmentation and storage in FIG. 2 ). can be distributed to In this case, for example, data encoded by the encoding unit 111 is input to the latter device through a communication path such as a network or a coaxial cable. The latter device may obtain the encoded data via the communication path.

Hereinafter, the configuration of the server 200 will be described. The server 200 distributes the distribution list and decision information to the clients 300 via the network 402 . The distribution list is a list in which information related to distributed data (hereinafter also referred to as content) is written. Typically, the content distributor writes metadata such as the content's acquisition destination and bit rate to the distribution list. The content acquirer acquires and analyzes the distribution list to specify the content to be acquired.

The contents listed in the distribution list include not only contents that can be transmitted, but also contents that are not ready to be transmitted and contents for which transmission is not permitted. The determination information is information for determining whether the receiving device (client 300) requests transmission of the content included in the distribution list.

As shown in FIG. 3 , the server 200 includes a detection unit 201 , a list generation unit 211 , a determination information generation unit 212 , a list transmission unit 213 , a determination information transmission unit 214 , a request a transceiver unit 215 , a distribution unit 216 , a data receiving unit 217 , a storage control unit 218 , a temporary storage 221 , and a storage 222 .

The detection unit 201 detects that the content is ready to be transmitted. For example, the detection unit 201 determines that the content is ready to be transmitted when the content is provided from the content providing device (eg, the transmission device 100 ). The detection unit 201 may monitor a storage area in which the content is stored (eg, the temporary storage 221 ) and determine that the content is ready to be transmitted when the content is stored.

The list generating unit 211 generates a distribution list. The list generating unit 211 generates the distribution list when requested by the client 300 to generate and transmit the distribution list, for example. The trigger for generating the distribution list is not limited thereto, and may be any trigger. For example, the list generating unit 211 may generate a distribution list of content to be transmitted in the next duration whenever a certain time elapses. The list generating unit 211 may generate the distribution list when the content is provided by the providing device or when instructed to generate the distribution list.

4 is a diagram illustrating an exemplary data structure of a distribution list. As shown in Fig. 4, the distribution list includes identification information for identifying content. 3 shows an example in which a uniform resource locator (URL) of content is used as identification information. The identification information may be any information other than a URL by which the content can be identified. The distribution list may include information other than identification information. In this embodiment, for example, a distribution list including the URL of each divided data (such as a picture) is generated.

As shown in FIG. 3 , the determination information generating unit 212 generates determination information. For example, when the content is ready to be transmitted, the determination information generating unit 212 generates determination information indicating that the content is ready to be transmitted, or when the content is not ready to be transmitted , the decision information generating unit 212 generates decision information indicating that the content is not ready to be transmitted. When the situation indicating whether the content is ready to be transmitted has changed, the determination information generating unit 212 generates updated determination information according to the changed situation.

The decision information generating unit 212 generates the decision information when requested by the client 300 to generate and transmit the decision information, for example. When the distribution list is generated by the list generating unit 211 , the decision information generating unit 212 may generate decision information in that state. The trigger for generating the judgment information is not limited to this, and may be any trigger. For example, whenever a certain time elapses, the determination information generating unit 212 detects whether the content is ready to be transmitted, for example, via the detection unit 201 , and is updated according to the detection result. Decision information can be generated.

5 is a diagram illustrating an exemplary data structure of decision information. The decision information of Fig. 5 is exemplary decision information for judging whether the four contents written in the distribution list of Fig. 4 are ready to be transmitted. For example, a circle indicates that the content is ready to be transmitted, and a cross indicates that the content is not ready to be transmitted. In the example shown in FIG. 5, four determination information (circle or cross mark) corresponding to four contents ("ContentA_1", "ContentA_2", "ContentA_3", and "ContentA_4") corresponding to the four URLs in FIG. ) is specified in an order corresponding to the order in FIG. 4 .

Decision information having a data structure other than that of Fig. 5 can be used in the case where the decision information can be specified for each content. For example, the decision information may be associated with identification information of the content. To identify which distribution list the decision information corresponds to, the decision information may be associated with information for identifying the distribution list.

The state in which the content is ready to be transmitted is, for example, the state in which the content to be distributed has been provided to the server 200 and is ready to be transmitted to the client 300 . The state in which the content is not ready to be transmitted is, for example, a state in which the content to be distributed is not provided to the server 200 and thus cannot be transmitted to the client 300 . The server 200 may transmit the content (pull-type transmission) according to a request from the client 300 , or transmit the content without a request from the client 300 (push-type). type) can be sent).

The server 200 may specify whether the content is ready to be transmitted regardless of whether the content is being presented. For example, in order to reduce the communication load, the server 200 generates, for some or all of the plurality of provided contents, determination information indicating that the contents are not ready to be transmitted, so as not to transmit these contents. can In this way, in some cases content that is ready to be transmitted is not ready to be transmitted.

The determination information may be provided as one or a plurality of pieces per content. For example, at least one metadata of the content may be used as decision information. The metadata is, for example, range information indicating the range of the content, the data length of the content, and the type of the content. The range information is, for example, information for specifying the range of a part of data to be distributed as content. When the range information is confirmed, the confirmed range information is set as the determination information, or when the range information has not yet been confirmed, predetermined information indicating that the range information has not yet been confirmed (unconfirmed information) is set as the determination information do. When the unconfirmed information is set in the range information as the determination information, the client 300 may determine that the corresponding content is not ready to be transmitted. In this way, when metadata is used as determination information, it is sufficient to set this data format to determine whether or not content can be transmitted based on the metadata.

As shown in FIG. 3 , the list sending unit 213 transmits the distribution list generated by the list generating unit 211 to the client 300 . For example, the list sending unit 213 transmits the distribution list to the client 300 in advance before starting the transmission of the content. The decision information sending unit 214 transmits the decision information generated by the decision information generating unit 212 to the client 300 .

The request transmitting/receiving unit 215 transmits/receives various requests. For example, the request transmission/reception unit 215 receives, from the client 300 , a transmission request of a distribution list, a transmission request of determination information, and a content transmission request. The request transmission/reception unit 215 transmits a transmission request of the content stored in the storage 121 to the transmission device 100 .

The distribution unit 216 transmits the requested content to the client 300 that has sent the transmission request. When push-type transmission is employed, the distribution unit 216 may transmit the content without a request from the client 300 .

The data receiving unit 217 receives data transmitted from the transmitting device 100 . For example, the data receiving unit 217 receives divided data as data to be transmitted to the server 200 . When the transmission request of the data stored in the storage 121 is transmitted, the data receiving unit 217 receives the data transmitted by the transmission device 100 in response to the transmission request.

Storage control unit 218 controls storage processing in temporary storage 221 and storage 222 . Data obtained by converting the data transmitted from the transmission device 100 and the data transmitted from the transmission device 100 into a format in which the data is to be distributed by the server 200 is stored in the storage 222 and then distributed. In this case, write processing abnormalities such as the processing speed of storing data in the storage 222 do not keep up with the distribution speed, and the writing processing in the storage 222 being temporarily frozen, occur, and in some cases Distribution cannot be performed normally. Accordingly, the storage control unit 218 stores the data transmitted from the transmission device 100 in the temporary storage 221 as a storage medium in which the above-described write processing abnormality does not occur. The distribution unit 216 reads data from the temporary storage 221 and distributes the data to the client 300 when the data is stored in the temporary storage 221 . In this case, data transmitted from the transmission device 100 is finally stored in the storage 222 . Accordingly, the dispensing unit 216 may operate as if dispensing is performed from the storage 222 in response to a request from the client 300 . Specifically, data stored in the temporary storage 221 may be returned in response to a request for data stored in the storage 222 . This functionality may be accomplished by recording the association of these files on the server 200 (eg, by using a database, file, or symbolic link) and referencing the association in distribution.

Next, the storage control unit 218 performs processing (write processing) of writing the data stored in the temporary storage 221 to the storage 222 . In the write processing, the storage control unit 218 may merge a plurality of data stored in the temporary storage 221 into one data and store the merged data in the storage 222 . For example, the storage control unit 218 may merge a plurality of data captured within a predetermined duration so that the data is included in one file, and write the merged file (merged file) to the storage 222 . Thus, it is possible to avoid a failure in which the number of files stored in storage 222 exceeds the number allowed by, for example, the operating system, and data can no longer be stored.

A plurality of data is merged into one data and stored in the storage 222 , and then, when a transmission request of data included in the merged data is received from the client 300 , the function of specifying data in the merged data is necessary. Thus, for example, the list generating unit 211 may update the distribution list so that the distribution list includes specific information for specifying data. For example, the list generating unit 211 creates a distribution list in which specific information (such as a byte offset from the beginning) indicating the location of data in the merge file is associated with identification information (such as a URL) of the data, and , and transmits the distribution list to the client 300 .

When referring to the updated distribution list and requesting transmission of data, the client 300 specifies identification information of the requested data and specific information associated therewith. For example, the client 300 requests data using information obtained by adding specific information to a URL indicating identification information of the data. The client 300 may request data through a transmission request having a header (eg, HTTP extension header) including specific information.

In this case, the requested file name may be the original file name before division. This facilitates the determination of whether data has already been received and cached on the client 300 and prevents the acquired data from being acquired again. In this case, the server 200 needs to specify the data (merged file) obtained by merging the original divided files (fragment files). This is the method (1) in which the information of the merge file for the fragment files is transmitted as specific information transmitted to the client 300 in advance, and written in the HTTP extension header upon request by the client 300, or the server 200 can be achieved by the method (2) of specifying the merge file from the requested fragment file names in response to the request. Method (2) records association with fragment files, for example (2-1) when a merge file is created (eg, by using a database, file, symbolic link), or (2-2) This can be done by automatically achieving the specification according to the naming convention (in the example shown in Fig. 4, the merge file names for ContentA_1 to ContentA_4 are ContentsA).

The server 200 specifies desired data from the merged data using the specific information transmitted from the client 300 , and distributes the specified data to the client 300 to which the request is sent.

The storage control unit 218 may delete data stored in the storage 222 according to a predetermined condition. For example, the storage control unit 218 may delete stored data for a certain duration. The storage control unit 218 may delete data in stages whenever a predetermined time elapses. For example, the storage control unit 218 deletes all or some P pictures in the storage 222 when a predetermined duration (eg, 1 day) has elapsed, and thereafter (equal to the first duration) or different) deletes the P pictures and I pictures in the storage 222 by progressively thinning each time a predetermined duration elapses. Through this processing, for example, when it is necessary to reduce the amount of storage for long-term recording, it is possible to obtain an image with increased intermittency with time without executing conversion processing into an image with reduced quality. . Stepwise deletion of P pictures may be performed by, for example, the encoding method disclosed in Japanese Patent No. 6239472.

For example, the storage control unit 218 may preferentially delete transmitted data. In this case, the storage control unit 218 may determine whether the data has been transmitted using metadata indicating whether the data has been transmitted. Also, the storage control unit 218 may prepare in advance other metadata to be referenced in the deletion process. This metadata is, for example, data for determining which picture the data corresponds to. More specifically, the metadata includes, for example, information indicating the time (eg, day, hour, minute, and second when it was taken), the position of the image within that time (eg, order from the beginning). ), the name of the file, and the byte offset from the beginning of the file.

The temporary storage 221 temporarily stores the received data. For example, the temporary storage 221 may be achieved by a temporary memory such as dynamic random access memory (DRAM).

The storage 222 stores various types of data to be used by the server 200 . For example, the storage 222 stores the content to be distributed, the generated distribution list, and the generated decision information. Storage 222 may be accomplished by any typically used storage medium, such as a memory card, random access memory (RAM), hard disk drive (HDD), or optical disk.

When the above-described write processing abnormality does not occur, no temporary storage 221 may be provided.

The above-mentioned components (detection unit 201, list generating unit 211, decision information generating unit 212, list sending unit 213, decision information sending unit 214, request sending/receiving unit 215, distribution unit Each of 216 , data receiving unit 217 , and storage control unit 218 ) is accomplished, for example, by one or more processors. For example, each component may be achieved by a computer program executed by a processor such as a CPU, ie, by software. A component may be achieved by a processor such as a dedicated IC, ie, by hardware. A component may be achieved by software and hardware. When multiple processors are used, each processor may achieve one of the components or may achieve two or more of the components.

The functions of the server 200 may be physically or logically distributed to a plurality of devices different from each other. For example, the functions may be distributed to a server device configured to transmit a distribution list and a server device configured to transmit content. Alternatively, for example, the functions receive data from the transmitting device 100 and a server device configured to store the data in the temporary storage 221 and the storage 222 , and the temporary storage 221 and the storage 222 . ) and distributed to a server device configured to distribute the data.

The following describes the functions of the client 300 . As shown in FIG. 3 , the client 300 includes a list receiving unit 311 , a decision information receiving unit 312 , a determining unit 313 , a request sending unit 314 , a data receiving unit 315 , and a reproducing unit. 316 and storage 321 .

The list receiving unit 311 receives the distribution list from the server 200 . The determination information receiving unit 312 receives determination information from the server 200 .

The determination unit 313 determines the content to be requested for transmission based on the distribution list and the determination information. For example, the judging unit 313 judges, among the contents whose URLs are described in the distribution list shown in FIG. 4 , the content whose circle is set as the determination information as shown in FIG. 5 as the content to be requested for transmission . When the above-described range information is used as the determination information, the determination unit 313 determines, for example, content for which unconfirmed information is not set in the range information, as the content to be requested for transmission. When a plurality of determination information is used, the determination unit 313 may determine the content to be requested for transmission according to the combination of determination information. For example, when all the determination information indicates that the corresponding contents are ready to be transmitted, the determination unit 313 determines the corresponding contents as the contents to be requested for transmission.

The request sending unit 314 transmits to the server 200 a transmission request of the content determined to be requested for transmission. The data receiving unit 315 receives, from the server 200 , the content transmitted according to the transmission request sent by the request sending unit 314 . The playback unit 316 plays the received content.

The storage 321 stores various types of data to be used by the client 300 . For example, the storage 321 stores the transmitted distribution list, the transmitted determination information, and the distributed content.

The above-described components (list receiving unit 311, determination information receiving unit 312, determination unit 313, request transmitting unit 314, data receiving unit 315, and reproducing unit 316) are each, This is accomplished, for example, by one or more processors. For example, each component may be achieved by a computer program executed by a processor such as a CPU, ie, by software. A component may be achieved by a processor such as a dedicated IC, ie, by hardware. A component may be achieved by software and hardware. When multiple processors are used, each processor may achieve one of the components or may achieve two or more of the components.

When the distribution list and determination information as described above are used, the server 200 does not need to create and transmit the distribution list every time there is an update, for example. The client 300 may more easily analyze the updated part by referring to the judgment information. Specifically, it is possible to efficiently acquire and analyze information equivalent to the latest distribution list without acquiring and analyzing the distribution list again.

The distribution list can be created and transmitted whenever there is a data update without using decision information, for example. In this case, the functions related to the decision information (eg, the decision information generating unit 212 , the decision information transmitting unit 214 , and the decision information receiving unit 312 ) need not be provided. For example, when push-type transmission is employed, the server 200 may distribute the image to the client 300 without using a distribution list.

Hereinafter, data transmission processing performed by the transmission device 100 according to the present embodiment will be described. 6 is a flowchart showing exemplary transmission processing in the present embodiment.

The imaging unit 101 takes an image to be distributed (step S101). The encoding unit 111 encodes the image input from the imaging unit 101 (step S102). The dividing unit 112 divides the image into data to be transmitted to the server 200 and data to be stored in the storage 121 . For example, the dividing unit 112 divides the input image into I-picture fragments and P-picture fragments (step S103). The data transmission unit 113 transmits the I-picture fragment to the server 200 (step S104). The storage control unit 114 stores the P picture fragment in the storage 121 (step S105).

The following describes a process for transmitting data stored in the storage 121 . 7 is a flowchart illustrating an exemplary process for transmitting data stored in storage 121 .

The request receiving unit 115 receives, from the server 200 (or the control server), a request for transmission of data stored in the storage 121 (step S201). When the P pictures are stored in the storage 121 , the request receiving unit 115 receives a request to transmit any of the stored P pictures. The data transmission unit 113 reads the requested P picture from the storage 121, and transmits the read P picture to the server 200 (step S202).

Hereinafter, list generation processing performed by the server 200 according to the present embodiment configured as described above will be described. In the list generation process, the server 200 creates a distribution list. Fig. 8 is a flowchart showing an exemplary list generation process in the present embodiment.

The list generating unit 211 of the server 200 generates, for example, a distribution list according to a request from the client 300 (step S301). The list generating unit 211 stores the generated distribution list, for example, in the storage 222 (step S302).

For example, the contents to be distributed are temporally continuous contents of ContentA_1, ContentA_2, ContentA_3, and ContentA_4, and ContentA_1 and ContentA_3 are distributable. In this case, the list generating unit 211 generates a distribution list for distributing ContentA_1, ContentA_2, ContentA_3, and ContentA_4. 4 above shows an exemplary distribution list created in this case.

Hereinafter, determination information generation processing performed by the server 200 according to the present embodiment will be described. In the decision information generation process, the server 200 generates decision information. Determination information generation processing is executed when, for example, a request for transmission of determination information is transmitted from the client 300 . Fig. 9 is a flowchart showing exemplary determination information generation processing in the present embodiment.

The determination information generating unit 212 of the server 200 determines whether determination information has been generated for the content to be distributed (step S401). When the determination information has not been generated (NO in step S401), the determination information generating unit 212 generates determination information for the content (step S402). The above-mentioned FIG. 5 shows exemplary determination information generated in the example described with reference to FIG. 8 (the example in which ContentA_1 and ContentA_3 are distributable).

After generating the determination information or when the determination information has been generated (YES in step S401), the determination information generating unit 212 determines whether to update the generated determination information (step S403). For example, upon receiving, from the detection unit 201 , a detection result indicating that the content is ready to be transmitted, the determination information generating unit 212 determines that the determination information should be updated.

When it is determined that the determination information should be updated (YES in step S403), the determination information generating unit 212 updates the determination information (step S404). After updating the decision information or when it is determined that the decision information will not be updated (NO in step S403), the decision information generating unit 212 ends the decision information generating process.

The server 200 maintains the generated distribution list and the generated determination information in a distributable state through the network 402 . The client 300 may access the server 200 to obtain the distribution list and determination information.

Hereinafter, content distribution processing performed by the communication system according to the present embodiment will be described. Fig. 10 is a sequence diagram showing exemplary content distribution processing in the present embodiment.

The list generating unit 211 of the server 200 generates a distribution list (step S501). This processing corresponds to, for example, the list generation processing described above. The list sending unit 213 of the server 200 transmits, for example, the distribution list to the client 300 according to a request from the client 300 (step S502).

The determination information generating unit 212 of the server 200 generates determination information (step S503). This processing corresponds, for example, to the above-described determination information generation processing. The determination information sending unit 214 of the server 200 transmits the determination information to the client 300 according to a request from, for example, the client 300 (step S504).

The list receiving unit 311 and the decision information receiving unit 312 of the client 300 receive the distribution list and the decision information, respectively. Then, the determination unit 313 of the client 300 determines the content to be requested for transmission based on the distribution list and thus received determination information (step S505). For example, when the distribution list shown in Fig. 4 is received, the determination unit 313 analyzes the received distribution list to obtain URLs for obtaining ContentA_1 to ContentA_4. The determination unit 313 also analyzes the received determination information to determine content ready to be transmitted among the contents written in the distribution list.

FIG. 11 is a diagram for explaining an example in which judgment is performed on the distribution list shown in FIG. 4 based on the judgment information shown in FIG. 5; FIG. 11 , the determination unit 313 can determine that ContentA_1 is ready to be transmitted (accessed) by combining the URL for obtaining ContentA_1 written in the distribution list and determination information corresponding to ContentA_1 . Similarly, the determination unit 313 may determine that ContentA_2 is not ready to be transmitted (accessed).

As shown in Fig. 10, the request sending unit 314 transmits a transmission request of the content determined to request transmission to the server 200 (step S506). The distribution unit 216 of the server 200 transmits the requested content to the client 300 (step S507). The data receiving unit 315 of the client 300 receives the content, and the reproducing unit 316 plays the received content (step S508).

When the content expressed in a format directly distributable from the server 200 is transmitted from the transmission device 100 , the distribution unit 216 directly transmits the content transmitted from the transmission device 100 to the client 300 without conversion. When the content is transmitted in a format additionally including metadata in which the content can be converted into a format distributed by the server 200, the distribution unit 216 converts the transmitted content into a distributable format according to the metadata. and transmits the converted content to the client 300 .

After that, for example, ContentA_2 became distributable. In this case, the detection unit 201 of the server 200 detects that ContentA_2 has become distributable. The decision information generating unit 212 generates updated decision information according to the detection result (step S509).

12 is a diagram illustrating exemplary updated decision information. As shown in Fig. 12 , the decision information generating unit 212 updates the decision information corresponding to ContentA_2 from the X table to the circle. The updated decision information remains distributable via the network 402 .

As shown in FIG. 10 , the determination information sending unit 214 of the server 200 transmits the updated determination information to the client 300 , for example, according to a request from the client 300 (step S510 ). ). The following steps S511 to S514 are the same as steps S505 to S508.

Fig. 13 is a view for explaining an example of making a judgment on the distribution list shown in Fig. 4 based on the judgment information shown in Fig. 12; 13 , the determination unit 313 may determine that ContentA_2 is ready to be transmitted (accessed).

In this way, when ContentA_2 becomes distributable, the client 300 can obtain the latest status of each content only by applying the updated judgment information to the distribution list already received without acquiring the distribution list again. .

This embodiment is applicable, for example, to a system for monitoring an image captured by a driving recorder. The system to which this embodiment is applicable is not limited to this system. For example, the present embodiment can be applied to a system for distributing and monitoring sensor data acquired by a sensor, and a system for distributing and monitoring moving image data acquired by an imaging device mounted on a moving object or the like. A moving object may be, for example, a person, robot, vehicle (car, two-wheeled vehicle, or electric train), truck, object capable of flying (manned airplane, unmanned aerial vehicle (eg, unmanned aerial vehicle (UAV), or drone)), or It is personal mobility. The moving object is, for example, a moving object that travels through a driving operation by a human, or a moving object that can be automatically driven (automatic driving) without a driving operation by a human.

The present embodiment is also applicable to, for example, a monitoring system that monitors an operation history of a screen (human machine interface (HMI)) of a device monitoring control system. The screen operation history may be obtained, for example, by a function of photographing and recording a screen displayed on the display device. The image data obtained in this way may be used instead of the image captured by the transmitting device 100 .

In the device monitoring control system, usually, a plurality of monitoring screens are used, and therefore, the monitoring system of the screen operation history needs to monitor the images of the plurality of monitoring screens by arranging them side by side in some cases. Usually, such a process of displaying a large number of images in parallel has a high processing load, but in this embodiment only divided data (eg, I pictures) can be displayed, so that an increase in processing load can be reduced. can

1st modification

The network 401 may include a plurality of communication lines, and a portion of the data (eg, I pictures) divided by the dividing unit 112 can be connected to any of the communication lines (communication line 401A). may be transmitted to the server 200 through the communication line, and the remaining data (eg, P pictures) may be transmitted to the server 200 through another communication line (communication line 401B) among the communication lines. 14 is a diagram showing an exemplary configuration of the image distribution system according to the first modified example configured as described above. For example, the data transmitting unit 113 may distribute I pictures via the communication line 401A by a live stream, and may transmit P pictures via the communication line 401B at selective timing.

2nd modification

The divided data may not be transmitted to the server 200 and may be stored in the storage 121 in the transmission device 100 in a divided manner. Thus, for example, only necessary pictures (eg, I pictures) can be read from the storage 121 and displayed. The processing load can be reduced by the method of displaying only I pictures rather than the method of displaying all pictures including P pictures. Accordingly, for example, the client 300 or the like having limited processing capacity may read only I pictures from the storage 121 of the plurality of transmission devices 100 and display the I pictures in parallel. In addition, the remaining P pictures and the like can be read out step by step as necessary to achieve a moving display.

In this case, similar to the storage control unit 218 of the server 200, the storage control unit 114 may delete data stored for a predetermined duration or delete data in stages whenever a predetermined time elapses. . For example, when the storage capacity is insufficient, instead of chronologically deleting all data including I pictures and P pictures, delete only P pictures and keep I pictures as data that can be displayed in a thinned manner can achieve long-term storage and low-capacity consumption of data.

3rd modification

When an image is divided into pictures and distributed, the size of the distribution list potentially increases. Accordingly, the list generating unit 211 can create a distribution list using a plurality of files in a hierarchical structure. For example, the list generating unit 211 may hierarchically configure distribution lists in units of year, month, day, hour, and the like. For example, the distribution list in the lowest layer includes identification information of each fragment data divided into a predetermined range of time. The next higher distribution list includes information for identifying at least one distribution list corresponding to a time included in a specific day. The next higher distribution list includes information for identifying at least one distribution list corresponding to a day included in a specific month. The next higher distribution list includes information for identifying at least one distribution list corresponding to a month included in a specific year.

4th modification

Fig. 15 is a diagram showing the outline of distribution processing performed by the image distribution system according to the fourth modification. The image distribution system according to the present modification further includes a control server 510 and a web server 520 . In this modification, the server 200 is configured as a web server that distributes images (images).

The control server 510 includes a control unit 511 . The control unit 511 controls the transmission device 100 according to the request transmitted from the client 300 . For example, the client 300 sends a transmit request to the control server 510 indicating whether the pictures are split and only some of them (eg, only I pictures) are transmitted or all pictures are transmitted. In response to the transmission request, the control unit 511 of the control server 510 transmits (uploads) some of the pictures to the server 200 in a divided manner or sends a control signal indicating whether to transmit all pictures to the server 200 It transmits to the transmission device 100 .

The transmitting device 100 operates in a switching method according to the control signal transmitted from the control server 510 , and transmits only some of the divided images to the server 200 or all pictures to the server 200 as described above in this embodiment. ) is sent to

In capture stream transmission (upload transmission), data divided as data to be transmitted to the server 200 (first data) is transmitted to the server 200 . When the capture file is saved (file output), the divided data as data to be stored in the storage 121 (second data) is not transmitted to the server 200 but is stored. As shown in FIG. 15 , the stored data may be copied offline to the server 200 .

The server 200 converts the data received by the data receiving unit 217 into a distributable format as needed (stream conversion), and stores the converted data in the storage 222 . All data copied offline may be converted (batch conversion) at once and stored in the storage 222 .

The storage 222 stores a picture in a distributable format (a fragment file), a distribution list, and the like. The server 200 functions as an image server that distributes pictures stored in the storage 222 . As described above, the data received by the data receiving unit 217 may be stored in the temporary storage 221 , and the data stored in the temporary storage 221 may be distributed by the distribution unit 216 .

The client 300 includes a web browser provided with a viewer. The web browser is provided with an image player for playing an image. The video player is achieved, for example, as an application compatible with HTML5.

The web server 520 is a server device having functions other than a function as an image server. For example, the web server 520 is a server device for providing a user interface (UI) for an external system different from the image distribution system.

In this way, the image distribution system according to the above-described embodiment can be achieved as a web system used on the Internet, for example.

As described above, in the image distribution system according to the present embodiment, the line load and the processing load can be reduced without degrading the quality.

Hereinafter, a hardware configuration of the transmission device according to the present embodiment will be described with reference to FIG. 16 . 16 is an explanatory diagram showing an exemplary hardware configuration of the transmission device according to the present embodiment.

The transmission device according to the present embodiment includes a control device such as a CPU 51, storage devices such as a read only memory (ROM) 52 and a RAM 53, and a communication I/F connected to a network to perform communication. 54, and a bus 61 connecting these components.

A computer program to be executed by the transmitting device according to the present embodiment is included in the ROM 52 or the like in advance and provided.

The computer program to be executed by the transmitting device according to the present embodiment is a computer readable medium such as a compact disc read only memory (CD-ROM), a flexible disk (FD), a compact disc recordable (CD-R), or a digital versatile disc (DVD). It may be recorded as a file in an installable or executable format on a possible recording medium, and may be provided as a computer program product.

The computer program to be executed by the transmitting device according to the present embodiment may be stored on a computer connected to a network such as the Internet, and may be provided by downloading through the network. The computer program to be executed by the transmitting device according to the present embodiment may be provided or distributed via a network such as the Internet.

The computer program to be executed by the transmitting device according to the present embodiment may cause the computer to function as each component of the above-described transmitting device. The computer can be achieved by the CPU 51 reading the computer program from the computer-readable storage medium onto the main storage device and executing the computer program.

Although specific embodiments have been described, these examples are presented by way of example only and are not intended to limit the scope of the invention. Indeed, the novel embodiments described herein may be embodied in a variety of different forms; In addition, various omissions, substitutions, and changes in the form of the embodiments described herein may be made without departing from the spirit of the present invention. It is intended that the appended claims and their equivalents cover such forms or modifications as fall within the scope and spirit of the present invention.

Claims (21)

A transmitting device, comprising:
dividing a plurality of transmission data to be transmitted into first data and second data;
transmit the first data during a first period to a server device configured to distribute the transmission data to a receiving device;
store in storage the second data and metadata indicating that the second data has not been transmitted to the server device;
receive, from the receiving device or the server device, a transmission request of the second data determined not to be transmitted to the server device by using the metadata;
and one or more processors configured to transmit the second data to the server device according to the transmission request during a second period of time after the first period. According to claim 1,
the transmission data is moving picture data including one or more picture data encoded by intra-frame prediction and one or more picture data encoded by inter-frame prediction;
the first data includes one or more picture data encoded by the intra-frame prediction;
and the second data includes one or more picture data encoded by the inter-frame prediction. According to claim 1,
the transmission data is moving picture data including one or more picture data encoded by intra-frame prediction and a plurality of picture data encoded by inter-frame prediction;
The first data includes at least one picture data encoded by the intra-frame prediction and a part of a plurality of picture data encoded by the inter-frame prediction,
The second data includes a remaining portion of the plurality of image data encoded by the inter-frame prediction. According to claim 1,
the transmission data is moving picture data including a plurality of picture data encoded by intra-frame prediction and one or more picture data encoded by inter-frame prediction;
The first data includes a part of a plurality of image data encoded by the intra-frame prediction,
The second data includes a remaining portion of the plurality of picture data encoded by the intra-frame prediction, and one or more picture data encoded by the inter-frame prediction. According to claim 1,
the transmission data is moving picture data including a plurality of picture data encoded by intra-frame prediction and one or more picture data encoded by inter-frame prediction;
The first data includes a part of a plurality of image data encoded by the intra-frame prediction,
and the processors change a part of a plurality of image data to be included in the first data among a plurality of image data encoded by the intra-frame prediction according to a band of a network in which the transmission data is to be transmitted. According to claim 1,
the transmission data is moving picture data including a plurality of picture data encoded by intra-frame prediction and one or more picture data encoded by inter-frame prediction;
The first data includes a part of a plurality of image data encoded by the intra-frame prediction,
and the processors change a part of a plurality of image data to be included in the first data among a plurality of image data encoded by the intra-frame prediction according to a request from the receiving device or the server device. According to claim 1,
and the processors receive, from a device comprising an encoder, the transmit data encoded by the encoder. 3. The method of claim 2,
and the processors encode the first data so that an encoding amount is different according to a band of a network in which the transmission data is to be transmitted. According to claim 1,
and the processors change the size of the first data according to a band of a network to which the transmission data is to be transmitted. According to claim 1,
and the processors delete the second data stored in the storage according to a predetermined condition. According to claim 1,
and the processors convert the first data and the second data into a format of data to be distributed by the server device. According to claim 1,
and the processors add, to the first data and the second data, metadata for converting into a format of data to be distributed by the server device. delete delete delete delete delete delete delete A transmission method comprising:
dividing a plurality of transmission data to be transmitted into first data and second data;
transmitting the first data for a first period to a server device configured to distribute the transmission data to a receiving device;
storing the second data and metadata indicating that the second data has not been transmitted to the server device in storage;
receiving, from the receiving device or the server device, a transmission request of the second data determined not to be transmitted to the server device by using the metadata; and
and transmitting the second data to the server device according to the transmission request during a second period after the first period. A non-transitory computer-readable recording medium storing a computer program including programmed instructions, comprising:
The instructions, when executed by a computer, cause the computer to:
dividing a plurality of transmission data to be transmitted into first data and second data;
transmit the first data during a first period to a server device configured to distribute the transmission data to a receiving device;
store in storage the second data and metadata indicating that the second data has not been transmitted to the server device;
receive, from the receiving device or the server device, a transmission request of the second data determined not to be transmitted to the server device by using the metadata;
and transmit the second data to the server device according to the transmission request during a second period after the first period.

Images