JP2019041328A - Medium processing unit, program and method - Google Patents

Abstract

To allow for video display of arbitrary layout in multiple display devices, by efficient processing.SOLUTION: The present invention relates to a medium processing unit which has medium signal synthesis means for synthesizing multiple inputted medium signals, medium signal input means for inputting medium signals, transmitted from multiple medium signal transmitters, to the medium signal synthesis means, medium signal transmission means for transmitting synthetic medium signal to multiple medium signal receivers, information input reception means for receiving input of medium signal processing condition information, including medium signal transmission source information, medium signal synthesis processing condition information, and medium signal transmission destination information, and medium signal processing instruction means for instructing the medium signal synthesis means, the medium signal input means, and the medium signal transmission means so as to perform processing according to the medium signal processing condition information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a media processing apparatus, program, and method, and can be applied to a system that processes video / audio data received from a surveillance camera connected via an IP network on a server and displays the video on a plurality of display devices.

  In a conventional monitoring system using a monitoring camera, data transmitted from the monitoring camera is transmitted as it is to a display device provided with a display and displayed in a layout (frame configuration) set on the display device side (see Patent Document 1).

  In the conventional monitoring system, there are cases where one camera image is displayed on one display device and cases where a plurality of camera images are displayed on one display device. The layout is set in advance on the display device side, and it is common to select from several predetermined patterns.

JP 2014-192713 A

  However, in the conventional monitoring system, there are systems that display the monitoring camera video displayed on a certain display device on a plurality of other display devices, or display different images on each of the plurality of display devices. There is a problem that the degree of freedom is low and the number of layouts that can be displayed simultaneously is limited.

  In view of the above-described problems, a media processing apparatus, program, and method capable of displaying video of an arbitrary layout on a plurality of display apparatuses with efficient processing are desired.

  The media processing apparatus according to the first aspect of the present invention is (1) media signal synthesis means for performing signal synthesis processing for generating a synthesized media signal by synthesizing a plurality of input media signals, and (2) transmitting the media signal. Media signal input means for inputting media signals transmitted from a plurality of media signal transmission devices having a media signal transmission processing unit to the media signal synthesis means, and (3) a synthesized media signal generated by the media signal synthesis means, A media signal transmitting means for transmitting to a plurality of media signal receiving devices having a media signal reception processing section for receiving the media signal; and (4) a source of a media signal to be supplied from the media signal input means to the media signal synthesizing means. Source information relating to a media signal transmission device and media signal synthesis performed by the media signal synthesis means Information input accepting means for accepting input of media signal processing condition information including synthesis processing condition information describing processing conditions and destination information related to a destination media signal receiving device set in the media signal transmitting means; (5) Media for instructing the media signal synthesis means, the media signal input means, and the media signal transmission means to perform processing according to the media signal processing condition information received by the information input acceptance means Signal processing instruction means.

  A media processing program according to a second aspect of the present invention comprises: (1) media signal synthesis means for performing signal synthesis processing for generating a synthesized media signal by synthesizing a plurality of inputted media signals; A media signal input means for inputting media signals transmitted from a plurality of media signal transmission devices having a media signal transmission processing unit for transmitting to the media signal synthesis means, and (3) a synthesized media generated by the media signal synthesis means Media signal transmitting means for transmitting a signal to a plurality of media signal receiving apparatuses having a media signal reception processing unit for receiving the media signal; and (4) a media signal to be supplied from the media signal input means to the media signal synthesizing means. Source information relating to a source media signal transmitting apparatus, and the media signal combining means Information for accepting input of media signal processing condition information including synthesis processing condition information describing conditions for media signal synthesis processing to be performed and transmission destination information related to a media signal receiving device of a transmission destination set in the media signal transmission means And (5) the media signal synthesizing unit, the media signal input unit, and the media signal transmission unit so as to perform processing according to the media signal processing condition information received by the information input reception unit. It functions as a media signal processing instruction means for giving an instruction.

  A third aspect of the present invention is a media processing method performed by a media processing device. (1) The media processing device includes media signal synthesis means, media signal input means, media signal transmission means, information input reception means, and media signal processing. (2) the media signal synthesis unit performs signal synthesis processing to generate a synthesized media signal by synthesizing a plurality of input media signals, and (3) the media signal input unit The media signal transmitted from a plurality of media signal transmitting devices having a media signal transmission processing unit for transmitting the media signal is input to the media signal synthesizing means, and (4) the media signal transmitting means is generated by the media signal synthesizing means. The composite media signal is received by a plurality of media signal receivers having a media signal reception processing unit for receiving the media signal. (5) the information input accepting means includes source information relating to a media signal transmitting apparatus that is a source of a media signal to be supplied from the media signal input means to the media signal synthesizing means, and the media signal synthesis. Receiving media signal processing condition information including synthesis processing condition information describing conditions of media signal synthesis processing performed by the means and destination information relating to a destination media signal receiving device set in the media signal transmitting means. And (5) the media signal processing instructing means performs the processing according to the media signal processing condition information received by the information input accepting means, the media signal synthesizing means, the media signal input means, and the An instruction is given to the media signal transmitting means.

  According to the present invention, it is possible to provide a media processing device, a program, and a method capable of displaying video of an arbitrary layout on a plurality of display devices with efficient processing.

It is the block diagram shown about the whole structure of the monitoring system which concerns on 1st Embodiment. It is explanatory drawing shown about the example of the frame structure of the screen in the composite media signal which the monitoring server (media composition part) which concerns on 1st Embodiment produces | generates. It is explanatory drawing shown about the example of the operation screen at the time of the monitoring server (media provision management part) which concerns on 1st Embodiment receiving the production | generation (setting which performs the new video delivery of a surveillance camera) of a media provision process part. . It is explanatory drawing shown about the example of the operation screen at the time of the monitoring server (display layout request | requirement receiving part) which concerns on 1st Embodiment receiving the input (selection) of frame structure information. It is explanatory drawing shown about the example of the operation screen at the time of the monitoring server (display layout request | requirement receiving part) which concerns on 1st Embodiment receiving the input (selection) of transmission source information. It is explanatory drawing shown about the example of the operation screen at the time of the monitoring server (display layout request | requirement receiving part) which concerns on 1st Embodiment receiving the input (selection) of transmission destination information. It is explanatory drawing shown about the example of the transition of the operation screen at the time of the monitoring server (display layout request | requirement part) which concerns on 1st Embodiment receiving the input (selection) of transmission destination information. It is explanatory drawing shown about the example of the frame emphasis process which the monitoring server (media composition part) which concerns on 2nd Embodiment performs. It is the block diagram shown about the whole structure of the modification (example which applied this invention to the conference server) of embodiment.

(A) First Embodiment Hereinafter, a first embodiment of a media processing device, a program, and a method according to the present invention will be described in detail with reference to the drawings. Hereinafter, an example in which the media processing device, program, and method of the present invention are applied to a monitoring server will be described.

(A-1) Configuration of the First Embodiment FIG. 1 is a block diagram showing the overall configuration of the monitoring system 1 according to this embodiment.

  The monitoring system 1 shown in FIG. 1 includes ten monitoring cameras 10 (10-1 to 10-10), a monitoring server 20 that processes media signals such as video captured by each monitoring camera 10, and a monitoring server And five display terminals 40 (40-1 to 40-5) that output media signals such as images processed at 20, and display layout instruction means 30 (30-1 to 30-3) that accept operations from the user. )have. In addition, the number of each element which comprises the monitoring system 1 is not limited. Moreover, in the monitoring system 1, the connection system (communication system) between each apparatus is not limited. In the monitoring system 1, each device may be connected by an IP network (Ethernet (registered trademark)), or directly connected (for example, using a video signal or audio signal transmission cable, a USB cable, or the like). In the following description, it is assumed that the devices are connected by an IP network (Ethernet).

  The specific configuration of the monitoring camera 10 is not limited. In this embodiment, the surveillance camera 10 will be described assuming that various cameras capable of transmitting media signals such as video signals in real time to the surveillance server 20 by IP communication are applied. In this embodiment, each surveillance camera 10 includes a microphone M and a camera C, and transmits a media signal including an audio signal (acoustic signal) captured by the microphone M and a video signal captured (captured) by the camera C. explain. In FIG. 1, each monitoring camera 10 is illustrated as a configuration including a microphone M and a camera C, but each monitoring camera 10 may be configured to transmit at least a video signal.

  The specific configuration of each display terminal 40 is not limited. In this embodiment, each display terminal 40 can apply, for example, a PC having a display and a speaker, or a display device (for example, a TV device having a LAN interface) that can be directly connected to an IP network. In this embodiment, each display terminal 40 includes a speaker and a display, outputs an audio signal (acoustic signal) included in the supplied media signal from the speaker, and outputs a video signal included in the supplied media signal from the display. A description will be given assuming that the output is made. In FIG. 1, each display terminal 40 is illustrated as including a display D and a speaker S, but each display terminal 40 may be configured to display and output at least a video signal.

  The specific configuration of each display layout instruction unit 30 is not limited as long as it can accept an operation from a user using a user interface such as a GUI screen or a command line. As each display layout instruction means 30, for example, a Web browser operating on a computer can be applied. In FIG. 1, the display layout instruction unit 30 and the display terminal 40 are illustrated as separate terminals, but the display layout instruction unit 30 is realized on one of the display terminals 40 (for example, installing a predetermined Web browser) May be realized).

  As shown in FIG. 1, the monitoring server 20 further includes a media processing unit 200 that processes media signals (media data) including video signals and audio signals. Note that the media signal processed by the monitoring server 20 of this embodiment is described as including a video signal and an audio signal (acoustic signal). However, only the video signal may be processed.

  The media processing unit 200 (monitoring server 20) is a computer (such as a PC or workstation) (not limited to one, but may be configured to be capable of distributed processing of a plurality of units) on a program (in the embodiment). It may be constructed by installing (including a media processing program), and even in that case, it can be functionally represented as shown in FIG. Also, some or all of the components of the media processing unit 200 may be constructed as dedicated hardware (for example, a dedicated semiconductor chip).

  In FIG. 1, a media processing unit 200 receives data signals 21 (21-1 to 21-1) that receive encoded media signals (media signals including video signals) from the monitoring cameras 10 (10-1 to 10-10). 21-10), a decoder 22 (22-1 to 22-10) for decoding a media signal (encoded media signal) received by the data receiver 21 (21-1 to 21-10), and data input A selection unit 23, a data output selection unit 24, three media provision processing units 26 (26-1 to 26-3), and a media provision management unit 25 are provided.

  The data input selection unit 23 performs a process of switching (distributing and supplying) the media signal supplied from the decoder 22 (22-1 to 22-10) in response to a request from each media provision processing unit 26. For example, in the state of FIG. 1, the data input selection unit 23 sends the decoders 22-1, 22-2, and 22-3 to the media provision processing unit 26-1 in response to a request from the media provision processing unit 26-1. , 22-4 are switched.

  The data output selection unit 24 performs a process of switching (distributing and supplying) a media signal transmitted by each media provision processing unit 26. Details of the data output selection unit 24 will be described later.

  The media provision management unit 25 performs processing for managing the media provision processing unit 26. For example, the media provision management unit 25 performs processing for generating the media provision processing unit 26 (for example, thread generation processing or resource allocation processing) in response to an instruction (user operation) from the display layout instruction unit 30. In other words, each media provision processing unit 26 is a component generated in software by the media provision management unit 25 in the media processing unit 200.

  The media provision management unit 25 generates one media provision processing unit 26 for one display layout instruction unit 30. In FIG. 1, since three display layout instruction units 30-1, 30-2 and 30-3 are arranged, media corresponding to each of the display layout instruction units 30-1, 30-2 and 30-3 is provided. The processing units 26-1 to 26-3 are in a generated state.

  The method by which the media provision management unit 25 receives an instruction from the display layout instruction unit 30 is not limited. For example, when each display layout instruction unit 30 is configured using a Web browser, the media provision management unit 25 mounts the content of the Web screen (GUI screen) provided to the display layout instruction unit 30 on the Web server program. This can be realized. For example, the media provision management unit 25 generates a media provision processing unit 26 (for example, a new setting for monitoring camera video signal distribution) when a predetermined URL is accessed from the display layout instruction unit 30 (Web browser). A Web screen (GUI screen) that can be received is provided to the display layout instruction means 30. Then, the media provision management unit 25 executes management processing such as generation of a new media provision processing unit 26 based on the contents received from the display layout instruction unit 30.

  Each media provision processing unit 26 synthesizes media signals from the monitoring camera 10 (decoder 22) in response to an instruction from the corresponding display layout instruction unit 30 and responds to an instruction from the corresponding display layout instruction unit 30. Processing for transmitting a synthesized media signal (hereinafter referred to as “synthesized media signal”) to the display terminal 40 is performed. Each media provision processing unit 26 can set a plurality of transmission destinations (display terminals 40) of the composite media signal according to the control of the display layout instruction unit 30. In this case, the display layout instruction means 30 performs processing for transmitting the same composite media signal to a plurality of transmission destinations (display terminals 40).

  Next, the internal configuration of each media provision processing unit 26 will be described.

  Each media provision processing unit 26 includes a media composition unit 101, a composition method instruction unit 102, an encoder 103, a display layout request reception unit 104, and a data transmission unit 105. Each media provision processing unit 26 is generated by the number of transmission destinations (display terminals 40) of the composite media signal. For example, in FIG. 1, since the display terminals 40-1, 40-2, and 40-3 are set as transmission destinations in the media provision processing unit 26-1, the display terminals 40-1, 40-2, and 40- 3 are generated corresponding to the data transmission units 105-1, 105-2, and 105-3.

  Hereinafter, a group of display terminals 40 to which the same composite media signal is supplied from one media provision processing unit 26 is referred to as a data transmission group 100. For example, the media provision processing unit 26-1 supplies the composite media signal to the three display terminals 40-1 to 40-3. Therefore, the display terminals 40-1 to 40-3 constitute one data transmission group 100-1. The number of display terminals 40 set in the data transmission group 100 may be one or more. In FIG. 1, since the supply destination of the composite media signal of the media provision processing unit 26-2 is only the display terminal 40-4, one data transmission group 100-2 is configured only by the display terminal 40-4. Has been. Similarly, in FIG. 1, since the supply destination of the composite media signal of the media provision processing unit 26-3 is only the display terminal 40-4, only one display terminal 40-4 supplies the data transmission group. 100-3 is configured.

  The display layout request receiving unit 104 receives and inputs an input of information (hereinafter referred to as “media signal processing condition information”) describing instruction contents (instruction contents according to a user operation) from the display layout instruction unit 30. Each component in the media provision processing unit 26 and the data input selection unit 23 are controlled so as to generate a composite media signal according to the content of the media signal processing condition information received. The media signal processing condition information includes information describing the synthesis conditions of the synthesized media signal (hereinafter referred to as “synthesis condition information”) and identification information (for example, each of the destinations of the synthesized media signal (display terminal 40)). Information (hereinafter referred to as “destination information”) in which a list of IP addresses, host names, etc. of the display terminal 40 is described is included. In other words, the transmission destination information processed by each media provision processing unit 26 is a list of display terminals 40 belonging to the data transmission group 100 corresponding to each media provision processing unit 26.

  The synthesis condition information includes, for example, information (hereinafter referred to as “frame configuration information”) describing the frame configuration of a screen (screen configured by one or a plurality of frames) constituting the composite media signal, and each frame. Information (hereinafter referred to as “transmission source information”) describing identification information (list of identification information) of the transmission source (monitoring camera 10 (decoder 22)) of the corresponding media signal is included. The frame configuration information specifies, for example, information that defines the area (range) of each frame constituting the screen, information that describes the position and shape of each frame (the shape of the outline of the frame), and a frame configuration pattern. It may be a code. Further, the format of identification information of each monitoring camera 10 (decoder 22) in the media processing unit 200 is not limited. Hereinafter, as an example, the identification information (ID) of the monitoring cameras 10-1 to 10-10 (decoders 22-1 to 22-10) is 1 to 10, respectively.

  The method in which the display layout request receiving unit 104 receives an instruction from the display layout instruction unit 30 is not limited. For example, similarly to the media provision management unit 25, the display layout request receiving unit 104 can be realized by mounting the content of the Web screen (GUI screen) provided to the display layout instruction unit 30 on the program of the Web server. it can. For example, the display layout request receiving unit 104 displays a Web screen (GUI screen) that can accept media signal processing condition information when a predetermined URL is accessed from the display layout instruction unit 30 (Web browser). Provided to the instruction unit 30 and acquires media signal processing condition information based on the input content of the Web screen.

  The method of switching the connection destination of the display layout instruction means 30 from the media provision management unit 25 to the media provision processing unit 26 (display layout request receiving unit 104) is not limited. For example, after receiving that the media provision processing unit 26 (Web server) generates the media provision processing unit 26, the media layout processing unit 26 (Web browser) displays the display layout of the generated media provision processing unit 26. The request receiving unit 104 (Web server) may change the connection (redirect to the URL corresponding to the generated media provision processing unit 26).

  The display layout request receiving unit 104 instructs the synthesis method instruction unit 102 to perform a media signal synthesis process according to the synthesis condition information of the media signal processing condition information.

  The synthesizing method instruction unit 102 supplies the media signal of the monitoring camera 10 (decoder 22) in which the identification information is set in the transmission source information of the synthesizing condition information to the media synthesizing unit 101 of the media provision processing unit 26. The switching of the data input selection unit 23 is set. Then, the synthesis method instruction unit 102 synthesizes the media signal supplied from the data input selection unit 23 according to the synthesis condition information (frame configuration information and transmission source information) of the media signal processing condition information. 101 is controlled.

  The media composition unit 101 controls the media signal supplied from the data input selection unit 23 (the media signal of the monitoring camera 10 (decoder 22) whose identification information is set in the transmission source information) by the composition method instruction unit 102 (composition). In accordance with the frame configuration information in the condition information), a media composite signal is generated and supplied to the encoder 103. In this embodiment, the media synthesis unit 101 is described as one that synthesizes a video signal and an audio signal (acoustic signal) out of media signals, but only the video signal may be synthesized. In this embodiment, the media compositing unit 101 performs processing for fitting a video of each monitoring camera 10 supplied to each frame of a frame configuration (screen configuration) according to the frame configuration information for the video signal. Shall. Also, the media synthesis unit 101 mixes various audio signals such as the audio signals supplied from the monitoring cameras 10 as they are, or the gain (power) is adjusted and mixed with a predetermined weight. Mixing techniques can be applied. Note that the media synthesizing unit 101 naturally does not perform the synthesis process of the audio signal when the supplied media signal does not include the audio signal.

  The encoder 103 encodes the supplied composite media signal by a predetermined method (the encoding method to be applied is not limited), and supplies the encoded composite media signal to the data output selection unit 24.

  The display layout request receiving unit 104 generates the data transmission units 105 for the number of pieces of identification information of the display terminals 40 set in the transmission destination information included in the media signal processing condition information. The identification information (for example, IP address or host name) of the display terminal 40 is set. Further, the display layout request receiving unit 104 receives the synthesized media signal (encoded synthesized media signal) supplied from the encoder 103 of the media provision processing unit 26 to the data output selection unit 24 and generates the data transmission unit 105. The switching setting of the data output selection unit 24 is performed so as to be supplied (distributed) to each. As a result, the encoded composite media signal output from the encoder 103 is supplied to each data transmission unit 105. Each data transmission unit 105 performs processing for transmitting the composite media signal (encoded composite media signal) supplied from the data output selection unit 24 to a set transmission destination (display terminal 40). Although the transmission method (communication protocol) of the composite media signal by the data transmission unit 105 is not limited, for example, a transmission method corresponding to real-time transmission such as RTP (Real-time Transport Protocol) may be applied. .

  For example, in the media provision processing unit 26-1 in FIG. 1, three data transmission units 105-1, 105-2, and 105-3 are generated. In addition, the data output selection unit 24 in FIG. 1 is configured so that the media signal (encoded synthesized media signal) supplied from the encoder 103 of the media provision processing unit 26-1 is the data transmission unit of the media provision processing unit 26-1. Switching settings are made so as to be supplied (distributed) to 105-1, 105-2, and 105-3. Further, in the media provision processing unit 26-1 of FIG. 1, the transmission destinations of the data transmission units 105-1, 105-2, and 105-3 are set to the display terminals 40-1, 40-2, and 40-3, respectively. ing.

  Next, details of the composition processing performed by the media composition unit 101 will be described.

  As described above, the media synthesis unit 101 synthesizes the media signal supplied from the data input selection unit 23 according to the control of the synthesis method instruction unit 102 (frame configuration information of the synthesis condition information), and generates a media synthesis signal. This is supplied to the encoder 103.

  The media composition unit 101 corresponds to, for example, various layouts as shown in FIG. In FIG. 2, symbols (F1 to F4) of the frames are attached to the frames arranged on the screen of the composite media signal.

  FIG. 2A shows a layout configuration in which frames F1 to F4 having the same area (equal area) are arranged on the screen of the composite media signal. FIG. 2B shows a layout configuration in which a frame F1 that occupies more than half of the screen area and three frames F2 to F that are divided into three equal areas are arranged on the screen of the composite media signal. Yes. FIG. 2C shows a layout configuration in which only one frame F1 is arranged on the screen of the composite media signal. The media composition performed by the media composition unit 101 may have a layout configuration in which only one frame is arranged on the screen as shown in FIG. For example, the media composition unit 101 performs media signal composition processing by switching to a different image of the monitoring camera 10 at a predetermined interval in one frame F1 (for example, rotating the images of the plurality of monitoring cameras 10). May be.

(A-2) Operation of the First Embodiment Next, the operation (media processing method according to the embodiment) of the monitoring system 1 of the first embodiment having the above configuration will be described.

  Hereinafter, an operation from when the media provision processing unit 26-1 is generated in accordance with an instruction from the display layout instruction unit 30-1 illustrated in FIG. 1 until the media signal synthesis process is started will be described.

  As described above, the media provision management unit 25 performs processing (for example, thread generation processing or resource allocation processing) for generating the media provision processing unit 26 in response to an instruction (user operation) from the display layout instruction unit 30. Do.

  Here, as an example, it is assumed that the display layout instruction unit 30-1 accesses a predetermined URL corresponding to the media provision management unit 25 of the monitoring server 20 by a user operation.

  When receiving the access from the display layout instruction unit 30-1 (Web browser), the media provision management unit 25 determines whether or not to newly generate the media provision processing unit 26 (for example, a new video of the monitoring camera 10). An operation screen (hereinafter referred to as a “new delivery acceptance screen”) for accepting whether or not to set delivery is provided.

  FIG. 3 is an explanatory diagram showing a configuration example of a new distribution acceptance screen.

  In the new distribution acceptance screen shown in FIG. 3, an OK button B101 for accepting a new generation of the media provision processing unit 26 together with a message “Do you want to set a new video distribution for the surveillance camera?” A Cancel button B102 for accepting that the media providing processing unit 26 is not generated is arranged.

  Here, it is assumed that the OK button B101 is pressed on the new distribution acceptance screen, and the media provision management unit 25 newly generates the media provision processing unit 26-1. Note that the number of initial data transmission units 105 in the media provision processing unit 26-1 may be one. Further, when the Cancel button B102 is pressed on the new distribution acceptance screen, the media provision management unit 25 ends without performing the subsequent processing.

  When the media provision management unit 25 generates the media provision processing unit 26-1, the URL of the connection destination of the display layout instruction unit 30-1 is the URL corresponding to the display layout request reception unit 104 of the media provision processing unit 26-1. To redirect (switch) to.

  Thereby, the connection destination of the display layout instruction unit 30-1 is the display layout request receiving unit 104 of the media provision processing unit 26-1.

  Upon receiving access from the display layout instruction unit 30-1, the display layout request reception unit 104 of the media provision processing unit 26-1 sends media signal processing condition information (synthesis condition information) to the display layout instruction unit 30-1. An operation screen (content) for requesting input of (frame configuration information and transmission source information) and transmission destination information) is provided. Note that the media signal processing condition information is configured by information including synthesis condition information and transmission destination information as described above. The synthesis condition information includes frame configuration information and transmission source information.

  FIG. 4 is an explanatory diagram showing a configuration example of an operation screen (hereinafter referred to as a “frame configuration information input screen”) that accepts input of frame configuration information constituting the synthesis condition information.

  As shown in FIG. 4, on the frame configuration information input screen, input of frame configuration information may be received by receiving an operation for selecting one of a plurality of frame configuration (layout configuration) candidates (patterns). Good.

  In FIG. 4, radio buttons RB201 to RB204 corresponding to four frame configurations (frame configuration patterns) and an OK button B201 for accepting input content determination are arranged. In this case, the display layout request receiving unit 104 selects (depresses) a radio button corresponding to one of the frame configurations on the frame configuration information input screen shown in FIG. 4 and then selects the OK button B201. The frame configuration information corresponding to the radio button is input as frame configuration information.

  Here, as shown in FIG. 4, the display layout request receiving unit 104 of the media provision processing unit 26-1 has selected the frame configuration corresponding to the radio button RB203 (the frame configuration shown in FIG. 2B). The following will be described.

  FIG. 5 is an explanatory diagram showing a configuration example of an operation screen (hereinafter referred to as “transmission source information input screen”) that accepts input of transmission source information constituting the synthesis condition information.

  In the transmission source information input screen shown in FIG. 5, for each frame of the frame configuration selected on the frame configuration information input screen, there is an input field for receiving input of the ID of the monitoring camera 10 (decoder 22) that is the transmission source of the media signal. Has been placed.

  In FIG. 5, input fields E301 to E304 for receiving ID inputs of the surveillance camera 10 (decoder 22) corresponding to the four frames F1 to F4, and an OK button B301 for receiving input content determination are arranged. .

  In this case, if the ID of the monitoring camera 10 is input to the input fields E301 to E304 and then the OK button B301 is pressed on the transmission source information input screen shown in FIG. The IDs input to 3401 to 3404 are recognized as the IDs of the monitoring cameras 10 corresponding to the fields F101 to F104, and the recognized information is acquired as transmission source information. In FIG. 5, since IDs 1, 2, 3, and 4 are respectively input in the input fields E301 to E304, the display layout request receiving unit 104 of the media provision processing unit 26-1 receives frames F1 to F4, respectively. Information indicating that the media signals of the monitoring cameras 10-1 to 10-4 are allocated (information in the table shown in FIG. 5) is acquired as transmission source information.

  FIG. 6 is an explanatory diagram showing a configuration example of an operation screen (hereinafter referred to as “transmission destination information input screen”) that accepts input of transmission destination information.

  In the transmission destination information input screen shown in FIG. 6, a display field D401 for displaying a list of transmission destinations of the composite media signal, an input field E401 for receiving input of the transmission destination, and a transmission field input in the input field E401 are display fields. An add button B402 for accepting processing to add to the list of D401 and an OK button B401 for accepting input of a destination list displayed in the display field D401 as destination information are arranged. That is, in the transmission destination information input screen shown in FIG. 6, when the transmission destination is input in the input field E401 and then the add button B402 is pressed, the transmission destination is added to the list of the display field D401.

  Hereinafter, the host names of the display terminals 40-1 to 40-5 will be described as HOST1 to HOST5. The identifier of the transmission destination in the transmission destination information may be other information such as an IP address or a MAC address.

  In the transmission destination information input screen of FIG. 6, HOST1 and HOST2 are already displayed (input) in the list displayed in the display field D401. In FIG. 6, since HOST3 is input in the input field E401, when the add button B402 is pressed in this state, HOST3 is added to the list of the display field D401 as shown in FIG. It becomes.

  When the OK button B401 is pressed in the state of FIG. 7, the display layout request receiving unit 104 of the media provision processing unit 26-1 displays the list of HOST1, HOST2, and HOST3 displayed (input) in the display field D401. It will be acquired as destination information.

  As described above, the display layout request reception unit 104 of the media provision processing unit 26-1 accepts input of synthesis condition information (frame configuration information and transmission source information) constituting the media signal processing condition information and transmission destination information. be able to.

  Then, when receiving the input of the media signal processing condition information, the display layout request receiving unit 104 of the media providing processing unit 26-1 controls each component so that the media signal is processed according to the media signal processing condition information. I do.

  First, the display layout request receiving unit 104 of the media provision processing unit 26-1 generates the data transmission units 105 for the number of transmission destinations registered in the transmission destination information constituting the media signal processing condition information, and each data The transmission destination (display terminal 40) of the transmission unit 105 is set.

  As described above, since “HOST1, HOST2, and HOST3” are set in the transmission destination information, the display layout request reception unit 104 of the media provision processing unit 26-1 has three types as shown in FIG. Data transmission units 105-1 to 105-3 are generated. Then, as shown in FIG. 1, the display layout request receiving unit 104 of the media provision processing unit 26-1 sends HOST1 to HOST3 (display terminal 40-1) to the transmission destinations of the data transmission units 105-1 to 105-3, respectively. To 40-3).

  Further, the display layout request receiving unit 104 of the media providing processing unit 26-1 supplies the synthesis method instruction unit 102 with the synthesis condition information constituting the media signal processing condition information, and the media signal according to the synthesis condition information. Command to perform composition processing.

  When the synthesizing method instruction unit 102 of the media providing processing unit 26-1 acquires the synthesizing condition information, the media signal of the monitoring camera 10 (decoder 22) described in the transmission source information constituting the synthesizing condition information is converted into the media providing process. The switching setting of the data input selection unit 23 is changed so as to be supplied to the media composition unit 101 of the unit 26-1. As described above, since “1, 2, 3, 4” is set as the transmission source information here, the composition method instruction unit 102 of the media provision processing unit 26-1 performs monitoring as shown in FIG. Switching setting of the data input selection unit 23 so that the media signals of the cameras 10-1 to 10-4 (decoders 22-1 to 22-4) are supplied to the media composition unit 101 of the media provision processing unit 26-1. To change.

  In addition, when the synthesis method instruction unit 102 of the media provision processing unit 26-1 acquires the synthesis condition information, the synthesis method instruction unit 102-1 supplies the synthesis condition information (frame configuration information and transmission source information) to the media synthesis unit 101 to be used as the synthesis condition information. A command to synthesize a media signal (generate a synthesized media signal) with the contents (conditions) according to the instruction is issued.

  Further, the synthesizing method instruction unit 102 of the media provision processing unit 26-1 sends the media signal (encoded media signal) output from the encoder 103 to each data transmission unit 105 (105-1 to 105-3). The switching of the data output selection unit 24 is set so as to be supplied and distributed.

  As described above, when the monitoring server 20 operates, the media signal of the monitoring cameras 10-1 to 10-4 based on the transmission source information is supplied to the media composition unit 101 of the media provision processing unit 26-1, and the media The synthesizing unit 101 synthesizes the media signal based on the synthesizing condition information to generate a synthesized media signal, and transmits the destination based on the destination information via the data transmitting units 105-1 to 105-3 (display terminal 40- 1 to 40-3), the composite media signal is distributed.

  Further, the display layout request receiving unit 104 of the media provision processing unit 26-1 uses the operation screen as shown in FIGS. 4 to 6 even when the media composition processing by the media composition unit 101 is being performed. The input of media signal processing condition information (synthesis condition information and transmission destination information) (change of input contents) may be accepted. That is, the display layout request receiving unit 104 receives media signal processing condition information (synthesis condition information and transmission destination information) input (change of input contents) in real time and newly receives an input (change) media signal processing condition. Each component may be controlled to perform media composition processing based on the information.

(A-3) Effects of First Embodiment According to the first embodiment, the following effects can be achieved.

  In the first embodiment, the following items (a) to (d) can be realized simultaneously.

  (A) The monitoring server 20 (media processing unit 200) of the first embodiment selects and supplies media signals from any combination of monitoring cameras 10 (decoders 22) to each media providing processing unit 26; Can be synthesized.

  (B) In the monitoring server 20 (media processing unit 200) of the first embodiment, the media synthesis unit 101 of each media provision processing unit 26 synthesizes a media signal (video signal) on a screen with an arbitrary frame configuration. Media signals (video signals) can be distributed to a plurality of transmission destinations (display terminals 40). Then, in the monitoring server 20 (media processing unit 200) of the first embodiment, an arbitrary frame configuration (a frame configuration according to the synthesis condition information) is displayed on the arbitrary display terminal 40 with the selected video images of the monitoring cameras 10. ). For example, in the monitoring server 20 (media processing unit 200) according to the first embodiment, only a specific portion in the video of the monitoring camera 10 is zoom-displayed by adding an item of synthesis condition information. It is also possible to specify things. Further, for example, in the monitoring server 20 (media processing unit 200) of the first embodiment, the monitoring camera 10 can be selected by changing the composition condition information from the display layout instruction unit 30 while the video is displayed on the display terminal 40. Changes in frame configuration, zoom display, etc. can be received in real time.

  (C) The monitoring server 20 (media processing unit 200) of the first embodiment can distribute media signals with different synthesis condition information for each data transmission group 100.

  (D) In the monitoring server 20 (media processing unit 200) of the first embodiment, the same video as the video displayed on a certain display terminal 40 can be simultaneously displayed on a plurality of other display terminals 40. In addition, in the monitoring server 20 (media processing unit 200) of the first embodiment, what composite condition information video is displayed on which display terminal 40 can be changed in real time.

  Assuming the above (d), it has been difficult for the prior art to simultaneously realize the above (a) to (c). In the monitoring server 20 (media processing unit 200) of the first embodiment, the composite media signal is generated by the common media combining unit 101 for the plurality of display terminals 40. (D) can be realized simultaneously.

  Conventionally, there is a system that realizes the above (b), but a system that delivers a plurality of camera images to a terminal as they are has a large bandwidth in a large-scale system (many cameras and many display devices). It was difficult to use because Conventionally, there is a system that re-synthesizes a plurality of camera images into one stream by the server's digital signal processor. However, the display layout and size of the images are determined in advance and can be changed freely during operation. It was difficult to cope with the function of zooming in on a specific part.

(B) Second Embodiment Next, a second embodiment of the media processing apparatus, program and method according to the present invention will be described in detail with reference to the drawings. Hereinafter, an example in which the media processing device, program, and method of the present invention are applied to a monitoring server will be described.

(B-1) Configuration and Operation of Second Embodiment The monitoring system 1 (monitoring server 20) of the second embodiment can also be shown using FIG.

  In the second embodiment, the processing content of the media composition unit 101 is different. In the first embodiment, once the synthesis condition information is supplied from the synthesis method instruction unit 102, the media synthesis unit 101 performs media signal synthesis processing according to the synthesis condition information until the next instruction is received. In the second embodiment, the media synthesis unit 101 dynamically (autonomously) changes the content of the video display in the synthesized media signal. For example, the media composition unit 101 may dynamically change the frame configuration of the screen in the composite media signal.

  The media composition unit 101 is not limited to the trigger for changing the content of the video display in the composite media signal. For example, the media composition unit 101 may perform processing for changing the display content of the video signal in the composite media signal (hereinafter referred to as “display content change processing”) in accordance with the content of the media signal to be synthesized. .

  For example, the media composition unit 101 analyzes the contents of media signals (video signal and audio signal (acoustic signal)) acquired from each monitoring camera 10 to determine a predetermined event (for example, failure, accident, criminal act, / Accident / criminal sign, etc.), and the display content change processing for the composite media signal may be performed in accordance with the content of the detected event and the transmission source (monitoring camera 10) of the detected media signal. Good.

  For example, for each monitoring camera 10, the media composition unit 101 may provide information (hereinafter referred to as “voice information”) indicating the voice of a person who may be present at the installation location of the monitoring camera 10, A database of information (hereinafter referred to as “face information”) indicating a person's face that may exist at the installation location is held, and voices that are not registered in the database from the media signal of any of the monitoring cameras 10 When a face image is detected, occurrence (event occurrence) of a failure (existence of a suspicious person) may be detected (detected) with respect to the monitoring camera 10. As the face information, a face image of a person who may be present at the installation location of the monitoring camera 10 or a feature amount of the face of the person may be applied. In addition, as voice information, voice of a person who may be present at the installation location of the monitoring camera 10 (sample voice) or a feature amount of the voice of the person (for example, information such as voiceprint or spectrum) is applied. It may be.

  Then, when detecting the occurrence of a failure (the presence of a suspicious person; an event), the media composition unit 101 may perform an enhanced display on the video (frame) of the monitoring camera 10 that has detected the failure.

  FIG. 8 is an explanatory diagram illustrating an example in which the media composition unit 101 according to the second embodiment performs display content change processing (video display change processing).

  FIG. 8A shows a screen in the composite media signal before the display content change process is performed (before a failure (event) occurs). In the screen shown in FIG. 8A, the sizes of the frames F1 to F4 are uniform. FIGS. 8B to 8D show examples of screens after the display content changing process is performed from the screen of FIG. 8A. The frames F1 to F4 are frames corresponding to the monitoring cameras 10-1 to 10-4, respectively.

  Here, it is assumed that a failure (event) is detected from the media signal of the monitoring camera 10-1 with the screen of the composite media signal in the state of FIG.

  In this case, as shown in FIGS. 8B to 8D, the media composition unit 101 displays the composite media signal by highlighting the display of the frame F1 corresponding to the monitoring camera 10-1 on the composite media signal screen. Perform content change processing. Hereinafter, a frame to be emphasized in the display content changing process (for example, frame F1 in the case of FIGS. 8B to 8D) is referred to as an “emphasis target frame”.

  In the screen shown in FIG. 8B, the area of the frame F1, which is the frame to be emphasized, is changed widely and the areas of the other frames F2 to F4 are changed compared to before the display content change processing (FIG. 8A). Narrowly changed.

  In addition, in the screen shown in FIG. 8C, as compared with the display content change process before (FIG. 8A), the entire region (full screen) is set as the region of the frame F1 that is the emphasis target frame. The area | region of F2-F4 is abbreviate | omitted.

  Further, in the screen shown in FIG. 8D, the emphasis process is performed on the frame frame of the frame F1, which is the emphasis target frame, as compared with the display content change process before (FIG. 8A). In FIG. 8D, for the sake of illustration, the frame of the frame F1 is thickened and hatched (hatched). The processing method for emphasizing the frame frame is not limited. For example, the color of the frame frame may be changed (for example, changed from black to red), or the frame line of the frame frame may be thickened. Can be mentioned.

(B-2) Effects of Second Embodiment According to the second embodiment, the following effects can be obtained in addition to the effects of the first embodiment.

  In the second embodiment, a single media composition unit 101 is used to transmit a composite media signal subjected to display content change processing (for example, frame enhancement processing) to a plurality of display terminals 40. Therefore, in the monitoring server 20 according to the second embodiment, even when the display content change process is dynamically performed, the single media composition unit 101 sharing the display content change process of the plurality of display terminals 40 having the same screen is effective. Can be executed automatically.

(C) Other Embodiments The present invention is not limited to the above-described embodiments, and may include modified embodiments as exemplified below.

  (C-1) In the first and second embodiments, the example in which the media processing system (media processing apparatus) of the present invention is applied to the monitoring system 1 (monitoring server 20) has been described. You may make it apply to.

  The monitoring server 20 shown in FIG. 1 has a one-way communication configuration in which the media signal acquired by the monitoring camera 10 is transmitted to the display terminal 40. The conference system 1A (conference server 20A) may be constructed. In other words, in the conference system 1A shown in FIG. 9, the monitoring camera 10 and the display terminal 40 in each of the above-described embodiments are both used as the conference terminal 10A (terminal including the camera C, the display D, the microphone M, and the speaker S). Has been replaced. In the conference system 1A shown in FIG. 9, the monitoring server 20 in each of the above embodiments is replaced with a conference server 20A that can perform bidirectional media signal processing. That is, in the conference system 1A (conference server 20A) shown in FIG. 9, the processing of media signals in each direction by the conference server 20A is the same processing as the monitoring server 20 of each of the above embodiments.

  For example, in the conference system 1A shown in FIG. 9, media signals (video signals and audio signals) transmitted from the conference terminals 10A-1 to 10A-4 are synthesized by the monitoring server 20 (synthesized by one media synthesis unit 101). Thus, a composite media signal is generated, and the composite media signal is transmitted to the three conference terminals 10A-5 to 10A-7.

  When the monitoring system 1 (monitoring server 20) of the second embodiment is applied to the conference system 1A (conference server 20A), the media signal to be combined is also used by the media combining unit 101 installed in the conference server 20A. The occurrence of an event is detected from (the media signals transmitted from the conference terminals 10A-1 to 10A-4), and display content change processing is performed. In this case, the media composition unit 101 installed in the conference server 20A detects speech (utterance) as an event from the audio signals of the conference terminals 10A-1 to 10A-4, and speaks (utterance) on the screen of the composite media signal. A frame corresponding to the conference terminal 10 </ b> A that has detected this may be processed as an emphasis target frame.

  For example, when the media composition unit 101 mounted on the conference server 20A synthesizes the media signals of the conference terminals 10A-1 to 10A-4, the frames F1 to F4 of the screen of the synthesized media signal are respectively conference terminals 10A-1 to 10A-1. A case corresponding to 10A-4 is assumed. At this time, when the media composition unit 101 detects a speech (utterance) from the audio signal supplied from the conference terminal 10A-1, the frame F1 corresponding to the conference terminal 10A-1 is emphasized on the screen of the composite media signal. Processing (for example, enhancement processing as shown in FIG. 8) is performed.

  21, 21-1 to 21-10... Data receiving unit, 22, 22-1 to 22-10 ... decoder, 23... Data input selection unit, 24 ... data output selection unit, 25. -1 to 26-3... Media provision processing unit 101... Media composition unit 102. Composition method instruction unit 103. Encoder 104 104 Display layout request reception unit 105, 105-1, 105-2, 105-3 Data transmission unit 200 Media processing unit 30, 30-1 to 30-3 Display layout instruction means 40, 40-1 to 40-5 Display terminal 30 Display layout instruction means

Claims (9)

Media signal synthesizing means for performing signal synthesis processing for synthesizing a plurality of input media signals to generate a synthesized media signal;
Media signal input means for inputting media signals transmitted from a plurality of media signal transmission devices having a media signal transmission processing unit for transmitting media signals to the media signal combining means;
Media signal transmitting means for transmitting the synthesized media signal generated by the media signal synthesizing means to a plurality of media signal receiving devices having a media signal reception processing unit for receiving the media signal;
Combining processing conditions describing transmission source information relating to a media signal transmitting device that is a transmission source of a media signal to be supplied from the media signal input unit to the media signal combining unit, and conditions for media signal combining processing performed by the media signal combining unit Information input accepting means for accepting input of media signal processing condition information including information and destination information related to the destination media signal receiving apparatus set in the media signal transmitting means;
Media signal processing instruction means for instructing the media signal synthesis means, the media signal input means, and the media signal transmission means so as to perform processing according to the media signal processing condition information received by the information input acceptance means. A media processing device comprising:   The synthesis processing condition information includes frame configuration information describing a frame configuration on a screen of a video signal constituting a composite media signal, and a transmission source of the media signal applied to each frame described by the frame configuration information. 2. The media processing apparatus according to claim 1, further comprising transmission source information describing information relating to the media signal transmission apparatus.   3. The media signal synthesizing unit arranges the video of the video signal constituting each supplied media signal in a frame according to the synthesis processing condition information on the screen of the synthesized media signal. The media processing device according to 1.   4. The media signal synthesizing unit performs display content change processing for changing a display content of a video on a screen for changing the content of the synthesized media signal according to the content of the supplied media signal. The media processing device according to 1.   When there is a media signal in which a predetermined event is detected among the supplied media signals, the media signal synthesizing unit changes the display corresponding to the frame corresponding to the media signal on the screen of the synthesized media signal to be emphasized. The media processing apparatus according to claim 4, wherein   The said media signal synthetic | combination means detects the said predetermined event, when face images other than a predetermined person are reflected in the video signal which comprises the supplied media signal. Media processing equipment.   6. The media signal synthesizing unit detects the predetermined event when a voice other than a predetermined person is included in an audio signal among the supplied media signals. 6. The media processing device according to 6. Computer
Media signal synthesizing means for performing signal synthesis processing for synthesizing a plurality of input media signals to generate a synthesized media signal;
Media signal input means for inputting media signals transmitted from a plurality of media signal transmission devices having a media signal transmission processing unit for transmitting media signals to the media signal combining means;
Media signal transmitting means for transmitting the synthesized media signal generated by the media signal synthesizing means to a plurality of media signal receiving devices having a media signal reception processing unit for receiving the media signal;
Combining processing conditions describing transmission source information relating to a media signal transmitting device that is a transmission source of a media signal to be supplied from the media signal input unit to the media signal combining unit, and conditions for media signal combining processing performed by the media signal combining unit Information input accepting means for accepting input of media signal processing condition information including information and destination information related to the destination media signal receiving apparatus set in the media signal transmitting means;
Media signal processing instruction means for instructing the media signal synthesis means, the media signal input means, and the media signal transmission means so as to perform processing according to the media signal processing condition information received by the information input acceptance means. A media processing program characterized in that it functions as a computer program. In the media processing method performed by the media processing apparatus,
The media processing apparatus includes media signal synthesis means, media signal input means, media signal transmission means, information input reception means, and media signal processing instruction means.
The media signal synthesizing means performs a signal synthesizing process for synthesizing a plurality of inputted media signals to generate a synthesized media signal,
The media signal input means inputs media signals transmitted from a plurality of media signal transmission devices having a media signal transmission processing unit for transmitting media signals to the media signal synthesis means,
The media signal transmitting means transmits the synthesized media signal generated by the media signal synthesizing means to a plurality of media signal receiving devices having a media signal reception processing unit for receiving a media signal,
The information input receiving means includes transmission source information relating to a media signal transmission apparatus that is a transmission source of a media signal to be supplied from the media signal input means to the media signal synthesis means, and media signal synthesis processing performed by the media signal synthesis means. The media signal processing condition information including the combination processing condition information describing the conditions and the transmission destination information related to the transmission destination media signal receiving apparatus set in the media signal transmission means is received,
The media signal processing instructing unit provides the media signal synthesizing unit, the media signal input unit, and the media signal transmitting unit so as to perform processing according to the media signal processing condition information received by the information input receiving unit. A media processing method characterized by performing an instruction.

Images