JP2020030223A - Collaborative display system - Google Patents

Abstract

To provide a collaborative display system characterized in that plural terminals each having a display part collaborate with one another to constitute a sole pseudo display.SOLUTION: A terminal 3 downloads motion picture data, which contains all-pixel display information on a content, from a server 2 (step S101), and receives entry of a seat number 5 from a user (step S102). Thereafter, the terminal 3 identifies a pixel position of a responsible pixel on the basis of the entered seat number and map data 20 (step S103), and produces output data of the responsible pixel on the basis of the identified pixel position (step S104).SELECTED DRAWING: Figure 4

Description

  The present invention relates to a cooperative display system in which a plurality of terminals having a display unit cooperate to configure a single pseudo display.

  2. Description of the Related Art Conventionally, in events such as concerts, spectators have cheered on singers using chemical lights or battery-powered penlights. In recent years, there is also a penlight that can be controlled via wireless communication (see Patent Document 1). According to the penlight described in Patent Literature 1, it is possible to simultaneously turn off, turn on, and change the color, thereby producing a sense of unity in the venue.

  Further, Patent Literature 2 discloses a cooperative system having higher versatility and in which each device performs unified operation. According to the cooperative system described in Patent Literature 2, the master unit transmits a signal, the cooperating device receives the signal, and repeatedly transmits the same signal to another cooperating device. We can convey signal to spread.

Japanese Patent No. 5910192 Japanese Utility Model Registration No. 3193196

  However, neither Patent Literature 1 nor Patent Document 2 describes that the exact position of the penlight or the cooperating device is grasped. Cannot be constructed.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a cooperative display system in which a plurality of terminals having a display unit cooperate to form a single pseudo-display. It is.

  According to a first aspect of the present invention, there is provided a cooperative display system in which a plurality of terminals having a display unit cooperate to constitute a single pseudo-display and display moving image content. A terminal capable of communicating with the terminal, the terminal stores moving image data including display information of all pixels of the content, the terminal accepts an input of a seat number, and the terminal And based on map data indicating the correspondence between the pixel position in the pseudo display and the step of specifying the pixel position of the pixel in charge of itself, the terminal, the terminal specified from the video data, A preparatory process including a step of extracting the display information related to a pixel position; and the terminal displays the display information according to a command from the server. Step a cooperative display system and executes a display processing, the containing. According to the first aspect, a plurality of terminals having a display unit can cooperate to configure a single pseudo display. In particular, if the venue has a fixed seat number, the pseudo display can be easily configured.

  The preparatory processing according to the first invention may further include a step of setting the terminal so that the terminal can immediately read out the extracted display information in chronological order. Thus, it is possible to prevent an increase in delay time until the start of display due to a difference in processing speed of each terminal. In general, decoding of a moving image imposes a heavy load on the CPU, and it is assumed that the time required to start display varies greatly due to a difference in processing speed. By preparing display data of one pixel in charge of display instead of moving image data of the total number of pixels in advance, it is possible to immediately display without decoding each time. As a result, it is possible to minimize the variation in the delay time of the display start.

  In addition, the advance preparation processing in the first invention further includes a step in which the terminal specifies an audio channel and a volume based on information obtained from the display information, and the terminal includes Setting the sound information based on the sound channel and the sound volume to be immediately readable in chronological order. This allows the terminal to immediately read out the sound information during the display processing.

  Further, in the display processing according to the first aspect, the server further includes a step of transmitting a display start instruction of a head frame of the moving image data to the terminal; and the terminal receives the display start instruction. Displaying the display information in chronological order from the first frame, the server transmitting a display synchronization command to the terminal at a predetermined interval, and the terminal receiving a display synchronization command at a predetermined frame before receiving the display synchronization command. When displaying the display information for several frames, displaying the display information for the next frame for a predetermined frame number after receiving the display synchronization command, and displaying the display information before displaying the display information for the predetermined frame number. Displaying the display information about the next frame for the predetermined number of frames immediately after receiving the synchronization command. . As a result, even if the moving image has a long playback time, it is possible to prevent the display from gradually shifting, and to suppress the occurrence of the display shift to the end.

  Further, in the display processing according to the first aspect, the terminal may output the sound information in synchronization with displaying the display information. Thus, the display on the pseudo display and the sound can be linked.

  A second invention is a cooperative display system in which a plurality of terminals having a display unit cooperate to form a single pseudo display and display moving image content, and a server capable of communicating with the terminal, Comprising a photographing device for photographing an area in a pseudo display, wherein the terminal stores a program for providing a function of transmitting identification information of the terminal to the server using a color or a light emission pattern; and A step of converting the identification information of the terminal into the color or the light emission pattern based on identification information light emission conversion data indicating the correspondence between the identification information and the color or the light emission pattern of the terminal; Displaying the color or the light emission pattern to be converted, and wherein the photographing device displays the display of the terminal included in the pseudo display. Photographing, the server analyzes the photographing result by the photographing device, and extracts the color or the light emitting pattern for each terminal; and the server extracts the color or the light emitting pattern to be extracted. Identifying a pixel position in the pseudo display based on the position, and the server identifies the identification information based on the extracted color or emission pattern with reference to the identification information light emission conversion data Step and the server, the step of transmitting the pixel position associated with the identification information to the terminal, the terminal receives the pixel position associated with the identification information, from the content, A step of extracting display information about the pixel position associated with the identification information of the own, Serial terminal, in accordance with instructions from the server, a cooperative display system and executes a display process including the step of displaying the display information. According to the second aspect, a plurality of terminals having a display unit can cooperate to form a single pseudo display. In particular, the assigned pixel can be mapped to each terminal even in a venue where a seat number is not defined.

  A third invention is a cooperative display system in which a plurality of terminals having a display unit cooperate to constitute a single pseudo-display and display moving image content, wherein a server capable of communicating with the terminal, A photographing device for photographing an area in a pseudo display, wherein the terminal inputs input information by a user, and the terminal outputs input information light emission conversion data indicating correspondence between the input information and a color or a light emission pattern. Converting the input information to be input into the color or the light emission pattern based on the: the terminal displays the color or the light emission pattern to be converted; and Photographing the display unit of the terminal included in the server, the server analyzes a photographing result by the photographing device, and Information collection including: extracting a color or the light emission pattern; and the server referring to the input information light emission conversion data and specifying the input information based on the extracted color or light emission pattern. A cooperative display system that performs a process. According to the third aspect, the server can simultaneously read the questionnaire results and the messages from the terminal in the pseudo display, so that the questionnaire results of tens of thousands of people can be aggregated at a high speed, and a plurality of messages can be obtained in real time. can do. Thereby, a moving image to be reproduced can be selected based on the questionnaire result. For example, video content with a large number of voting results can be preferentially reproduced. In addition, the questionnaire results can be used for effects such as gender, age, and a display that distinguishes who the fan is. For example, the display content can be changed for each group of the seats (for example, the left side and the right side) based on information such as gender.

  According to the present invention, it is possible to provide a cooperative display system in which a plurality of terminals having a display unit cooperate to configure a single pseudo display.

The figure which shows the outline | summary of the cooperation display system 1a in 1st Embodiment. Diagram showing an example of download data The figure which shows an example of the output data 50 Flowchart showing the flow of the preparatory processing in the first embodiment Flowchart showing the flow of display processing in the first embodiment FIG. 4 is a diagram illustrating a synchronization process according to the first embodiment. The figure which shows the outline | summary of the cooperation display system 1b in 2nd Embodiment. The figure which shows an example of the identification information light emission conversion data 60 Flowchart showing the flow of the preparatory processing in the second embodiment The figure which shows an example of the input information light emission conversion data 70 Flow chart showing the flow of a questionnaire totalizing process in the second embodiment

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the cooperative display system of the present invention, a plurality of terminals having a display unit cooperate to form a single pseudo display. On the pseudo display, for example, content such as a moving image is displayed.

<First embodiment>
FIG. 1 is a diagram illustrating an outline of a cooperative display system 1a according to the first embodiment. The pseudo display 4 in FIG. 1 is not a display configured with a single housing, but a pseudo display configured with display units of a plurality of terminals 3 having different housings. The terminals 3 are arranged apart from each other. FIG. 1 schematically illustrates a part of a spectator seat of a venue where an event such as a concert is held as a space where the pseudo display 4 is configured. The spectator has a terminal 3 such as a smartphone, for example, and sits on a seat having a predetermined seat number 5. Numbers such as A1 and A2 shown in FIG.

  The cooperative display system 1a includes a plurality of terminals 3 and a server 2 communicably connected to each terminal 3. Although the communication means is not particularly limited, for example, a short-range wireless communication technology such as BLE (Bluetooth (registered trademark) Low Energy) or WiFi may be used within the venue. Further, the terminal 3 and the server 2 may communicate not only directly but also via a relay terminal having a relay function. Further, when exchanging data in advance outside the venue, the terminal 3 and the server 2 may communicate via the Internet. However, when there are a large number of terminals 3 (for example, tens of thousands) in a performance hall such as a concert or the like, if the individual terminals 3 and the server 2 perform individual communication, the communication line is broken. Therefore, in such a case, it is general that the individual terminal 3 and the server 2 do not perform individual communication but use a communication method (broadcast: broadcast) in which the server 2 side unilaterally distributes information.

  Each terminal 3 is arranged indoors or outside at night where no external light enters, and is in charge of displaying a part of the content displayed on the pseudo display 4. In the present embodiment, each terminal 3 is in charge of a display process for one pixel, and a plurality of terminals 3 cooperate to realize a single pseudo display 4. When the display on the pseudo display 4 is performed, by turning off the lighting in the venue as much as possible, the RGB values output to the display units of the plurality of terminals 3, that is, the collection of light spots, is displayed as if it were a dot picture. appear. In FIG. 1, a plurality of terminals 3 cooperate and display the numeral “1” on the pseudo display 4.

  In FIG. 1, 35 terminals 3 are illustrated, but the number of terminals 3 is not limited. Depending on the number of spectators at the event, several to tens of thousands of terminals 3 can constitute a single pseudo-display 4. Further, a plurality of pseudo displays 4 may be set in one venue. For example, when the audience seats are arranged around the stage, the pseudo display 4 may be set for each of the four directions viewed from the stage.

  The server 2 and the terminal 3 include a CPU (short for “Central Processing Unit”) as a control unit, a memory as a main storage, an HDD (short for “Hard Disk Drive”) or flash memory as an auxiliary storage, and an external storage. It has a storage medium as a unit, a liquid crystal display as a display unit, a keyboard and mouse as an input unit, a touch panel display, a wireless module as a wireless communication unit, and the like. An HDD (abbreviation of “Operating System”), an application program, data necessary for processing, and the like are stored in an HDD or a flash memory as an auxiliary storage unit. The CPU of the terminal 3 and the CPU of the server 2 read the OS and the application program from the auxiliary storage unit, store them in the main storage unit, control other devices while accessing the main storage unit, and execute processing described later. Further, the terminal 3 may include a speaker or the like as a sound output unit.

  The server 2 may be composed of one server or a plurality of servers. The server 2 connected to the terminal 3 via the Internet (for example, a server corresponding to downloading of an application or content of a smartphone) may be installed in a data center outside the venue. On the other hand, the server 2 connected to the terminal 3 via short-range wireless communication such as BLE is installed in the venue. Hereinafter, for the sake of simplicity, it is assumed that the server 2 is not distinguished depending on the installation location.

  FIG. 2 is a diagram illustrating an example of the download data. The download data is common data stored in the server 2 in advance and distributed to each terminal 3. In the first embodiment, the download data includes, for example, moving image data 10, map data 20, hue / sound conversion data 30, and brightness / volume conversion data 40. The hue sound conversion data 30 and the brightness / volume conversion data 40 are used when the terminal 3 outputs sound information and controls switching of sound information based on display information. Each terminal 3 downloads these data from the server 2 in advance.

  The moving image data 10 includes display information of all pixels of the content displayed on the pseudo display 4. The display information included in the moving image data 10 is, for example, an RGB value for each pixel. The moving image data 10 may include sound information. Further, the moving image data 10 may be a single moving image file or may be divided into a plurality of moving image files.

  The map data 20 is used for mapping the assigned pixel to each terminal 3. The map data 20 includes, for example, the seat number 21 of the seat included in the pseudo display 4 and the pixel position 22 of the content corresponding to the seat number 21. The seat number 21 is a data item having the same meaning as the seat number 5 shown in FIG. The pixel position 22 indicates the position of each pixel of the content by an X coordinate and a Y coordinate. The terminal 3 can determine the assigned pixel based on the seat number 21 by referring to the map data 20.

  When a plurality of pseudo displays 4 are set, a plurality of different moving image data 10 and map data 20 may be distributed for each pseudo display 4.

  The hue sound conversion data 30 is used for controlling the switching of the sound channel 32 by the hue 31, and the hue 31 and the sound channel 32 are associated with each other. As the hue sound conversion data 30, for example, the hue 31 specified by the RGB values may be divided into two ranges and associated with the left and right (L, R) sound channels 32. In addition, as a method of specifying a hue from an RGB value, an existing color model can be used. Here, the hue 31 is taken as an example, but the information associated with the audio channel 32 is not limited to this example, and any information can be obtained from the display information. Information is fine. For example, the terminal 3 may use the LSB (least significant bit) of each of RGB included in the moving image data or the LSB of the luminance value as a signal for switching the audio channel independently of the display.

  The brightness / volume conversion data 40 is used to control switching of the volume 42 by the brightness 41, and the brightness 41 and the volume 42 are associated with each other. For example, the brightness / volume conversion data 40 may be associated so that the volume increases as the brightness 41 specified by the RGB values increases. An existing color model can be used as a method of specifying the luminance from the RGB values. Here, the brightness 41 is taken as an example, but the information associated with the volume 42 is not limited to this example, and any information that can be obtained from the display information will be described. But it is good.

  The terminal 3 can convert the RGB values into an audio channel and an audio volume by using the hue audio conversion data 30 and the luminance volume conversion data 40, and can control the display and the audio in synchronization. As a result, the display on the pseudo display 4 and the audio can be synchronized. Even if the volume of one terminal 3 is low, by gathering tens of thousands of terminals 3, for example, it is possible to produce an effect in which the sound moves around in conjunction with the movement of the light spot.

  FIG. 3 is a diagram illustrating an example of the output data 50. The output data 50 is chronological display information and sound information output from each terminal 3 and is generated based on the moving image data 10 downloaded in advance by each terminal 3. A specific generation method will be described later with reference to FIG. The output data 50 includes an RGB value 51, a volume 52, an audio channel 53, and the like. The RGB value 51 is display information to be displayed on the display unit of the terminal 3. The sound volume 52 is one of the sound information to be output to the sound output unit of the terminal 3 and is the sound volume of the sound output from the speaker of the terminal 3. The audio channel 53 is one of the sound information, and is a number for identifying the audio channel of the sound output from the speaker of the terminal 3.

  The display information is not limited to the RGB value 51, and may be any information such as a gray scale value or an ON / OFF value related to a specific hue, as long as it is related to the pixel value of each pixel.

  FIG. 4 is a flowchart illustrating the flow of the preparatory processing according to the first embodiment. As shown in FIG. 4, the terminal 3 downloads an application and data relating to the cooperative display system 1a from the server 2 (Step S101), and installs the application. The data to be downloaded is the data shown in FIG.

  Next, the terminal 3 receives an input of the seat number 5 from the user via a touch display or the like (Step S102). The input receiving means for the seat number 5 is not limited to this example. For example, the terminal 3 may scan a barcode having information of the seat number, or may acquire the seat number from the electronic ticket.

  Next, the terminal 3 specifies the pixel position of its assigned pixel based on the seat number and the map data 20 (Step S103). Specifically, the terminal 3 refers to the map data 20 and specifies the pixel position 22 corresponding to the seat number input in step S102.

  Next, the terminal 3 generates output data of the responsible pixel based on the pixel position 22 specified in Step S103 (Step S104). Specifically, the terminal 3 extracts the RGB values related to the pixel position 22 of the assigned pixel from the moving image data 10 in the order of frame numbers (= time-sequential order), and sets the extracted data as output data 50. When outputting the sound information, the terminal 3 may extract the data relating to the volume 52 and the sound channel 53 from the moving image data 10 and store the data in a file independent of the file in which the RGB values 51 are stored. Alternatively, the terminal 3 specifies an audio channel from the moving image data 10 based on the hue relating to the RGB value using the above-described hue audio conversion data 30, and uses the luminance / volume conversion data 40 to specify the RGB channel. The volume may be specified based on the luminance and stored as a one-channel (monaural) audio signal. The information for specifying the audio channel is not limited to the hue, and may be any information as long as it is information obtained from the display information. For example, the terminal 3 may use the LSB (least significant bit) of each of RGB included in the moving image data or the LSB of the luminance value as a signal for switching the audio channel independently of the display. Similarly, the information for specifying the sound volume is not limited to the luminance, and may be any information as long as the information can be obtained from the display information.

  The terminal 3 can display the display information for one pixel on the pseudo display 4 and output the sound information by generating the output data 50 of the pixel in charge of the terminal 3 itself.

  Next, the terminal 3 stores the output data 50 of the assigned pixel in the main storage unit (step S105). Note that the terminal 3 may store the output data 50 in the auxiliary storage unit by dividing the output data 50 into small files in addition to making it resident in the main storage unit. In any case, the terminal 3 is in a state where the output data 50 of the assigned pixel can be immediately read out during the display processing. Similarly, the terminal 3 is in a state where sound information based on the volume and the audio channel specified in step S104 can be immediately read out in chronological order.

  When the content is a moving image, the entire data size may be several hundred MB to several GB. However, when thousands to tens of thousands of terminals 3 share the output data of the assigned pixel of each terminal 3 Since 50 is less than several MB, the data size is such that it can be resident in the main storage unit. Note that each terminal 3 may have the output data 50 resident in the main storage unit for each short moving image of several seconds.

  Since the load of the CPU is heavy in the decoding processing of the moving image, if each terminal 3 executes the decoding processing in real time, the variation in the response until the start of the display increases due to the difference in the performance of the terminal 3. In some cases, a variation of about one second occurs. Therefore, by decoding the moving image in advance and converting it into information for each pixel as in the output data 50 shown in FIG. 3, it is possible to minimize variations.

  When sound information is controlled using the hue / speech conversion data 30 and the luminance / volume conversion data 40, the terminal 3 causes only the RGB values to be resident in the main storage unit as output data 50, and performs real-time RGB processing when performing display processing. The value may be converted to sound information.

  The terminal 3 finally needs only the output data 50 of the pixel in charge of itself. Therefore, after the user arrives at the venue, the terminal 3 may transmit the seat number to the server 2, and the server 2 may generate and transmit the output data 50 for each terminal 3. However, when the number of terminals 3 becomes tens of thousands, the load on the server 2 is concentrated in the same time zone. Therefore, in the present embodiment, the terminal 3 downloads common data from the server 2 in advance, and the terminal 3 generates necessary data. Thereby, the load on the server 2 can be distributed over time.

  FIG. 5 is a flowchart illustrating the flow of the display process according to the first embodiment. FIG. 5 shows the processing contents of the terminal 3. As a premise of the display processing shown in FIG. 5, it is necessary to complete the preliminary preparation processing shown in FIG. Then, the terminal 3 displays the display information prepared in the preparatory processing according to the command from the server 2.

  Hereinafter, it is assumed that all the terminals 3 execute the display processing of the output data 50 at the same frame rate. Further, in order to avoid a shift in display processing for each terminal 3, the output data 50 is divided into a plurality of sets at predetermined intervals, and the server 2 transmits a synchronization command to all the terminals 3 at the timing of switching the sets. And For example, when the frame rate is 30 fps and the predetermined interval is 1 second, one set corresponds to 30 frames. When performing communication via BLE, the server 2 transmits a synchronization command by broadcast.

  When the moving image data 10 is divided into a plurality of files, the chronological order of the moving image files may be specified by, for example, adding a serial number to the end of the moving image file number, and the plurality of moving image files may be continuously reproduced. it can.

  As illustrated in FIG. 5, the terminal 3 sets the first frame of the first moving image file as a display target frame (step S201). When there are a plurality of contents, before step S201, the terminal 3 may receive, from the server 2, a content specifying command for specifying which content is to be displayed, and determine the content to be displayed. .

  Next, a display start command is received from the server 2 (step S202). When performing communication via BLE, the server 2 transmits a display start command by broadcast.

  Next, the terminal 3 displays the pixel value of the current display target frame (= the RGB value 51 of the output data 50) on the display unit (Step S203). The terminal 3 may output the sound information based on the volume 52 and the sound channel 53 of the output data 50 to the sound output unit in synchronization with displaying the pixel value of the current display target frame on the display unit.

  Next, the terminal 3 checks whether or not the final frame of the final moving image file has been reached (step S204). When the last frame of the last moving image file has not been reached (No in step S204), the terminal 3 sets the next frame as a display target frame (step S205), and increases the synchronization counter by 1 (step S206).

  Next, the terminal 3 checks whether a synchronization command has been received from the server 2 (step S207). If the synchronization command has been received (Yes in step S207), the terminal 3 proceeds to step S210. If the synchronization command has not been received (No in step S207), the terminal 3 proceeds to step S208.

  In step S208, the terminal 3 checks whether the synchronization counter has reached a predetermined number, that is, the number of frames included in one set. If the synchronization counter has not reached the predetermined number (No in step S208), the terminal 3 repeats from step S203. On the other hand, when the synchronization counter has reached the predetermined number (Yes in step S208), the terminal 3 waits until a synchronization command is received from the server 2 (step S209), and upon receiving the synchronization command, proceeds to step S210.

  In step S210, the terminal 3 resets the synchronization counter. Then, the terminal 3 sets the first frame of the next set (which may be the first set of the next moving image file) as the display target frame (step S211), and repeats from step S203. When the display target frame reaches the final frame of the final moving image file (Yes in step S204), the terminal 3 ends the processing.

  FIG. 6 is a diagram illustrating the synchronization processing according to the first embodiment. As shown in FIG. 6A, when the synchronization processing is not performed, in the case of a moving image having a long reproduction time, even if the display timing is adjusted by the display start command in the first frame, the processing speed of the terminal 3 gradually increases. The display shift becomes large. Therefore, as shown in FIG. 6B, in the present embodiment, a synchronization process is performed, and the server 2 periodically transmits a synchronization command. Then, like the terminal 3a shown in FIG. 6B, the terminal 3 having a low processing speed, that is, the terminal 3 receiving the synchronization command before the display of all the frames included in one set ends, The display of the frames included in the set is suspended, and the display of the first frame of the next set is performed. On the other hand, like the terminal 3b shown in FIG. 6B, the terminal 3 which has a high processing speed, that is, the terminal 3 which finishes displaying all the frames included in one set before receiving the synchronization command, Waits until receiving the next frame without displaying the next frame. Thus, even for a moving image having a long reproduction time, it is possible to prevent the display from gradually shifting, and to minimize the display shift to the end.

  As described above, according to the cooperative display system 1a in the first embodiment, a plurality of terminals 3 having a display unit can cooperate to configure a single pseudo display 4. In particular, in the first embodiment, the user inputs the seat number, and the terminal 3 refers to the predetermined map data 20 to map the assigned pixel to each terminal 3. Can be configured.

<Second embodiment>
FIG. 7 is a diagram illustrating an outline of a cooperative display system 1b according to the second embodiment. In the first embodiment, the assigned pixel is mapped to each terminal 3 using the seat number 5 capable of specifying the position of each terminal 3. However, in the second embodiment, the assigned pixel in the venue where the seat number 5 cannot be used is Provides a mapping method. Note that the same reference numerals are given to the same elements as those in the first embodiment, and redundant description will be omitted.

  The cooperative display system 1b captures images of the plurality of terminals 3, the server 2 communicably connected to each terminal 3, and the display units of all the terminals 3 included in the pseudo display 4, and enables communication with the server 2. And an imaging device 6 to be connected. The server 2 and the photographing device 6 are connected by a cable such as a USB cable or an HDMI (registered trademark) cable.

  Also in the cooperative display system 1b, each terminal 3 is in charge of one pixel of display processing, and a plurality of terminals 3 cooperate to realize a single pseudo display 4. In the example illustrated in FIG. 7, the plurality of terminals 3 cooperate and display the numeral “2” on the pseudo display 4.

  The image capturing device 6 is a video camera that captures a moving image at a predetermined frame rate, and is installed at a position where a moving image with image quality that can identify light emission of the display unit of each terminal 3 is obtained. The photographing device 6 may be constituted by one unit or a plurality of units.

  FIG. 8 is a diagram illustrating an example of the identification information light emission conversion data 60. The identification information light emission conversion data 60 is stored in the storage units of the server 2 and the terminal 3 in advance. In the second embodiment, predetermined identification information light emission conversion data 60 is used to map a responsible pixel to each terminal 3.

  The identification information light emission conversion data 60 includes, for example, an RGB value 62 and a light emission pattern 63 for each identification number 61 for identifying the terminal 3. The RGB value 62 is color information to be displayed on the display unit of the terminal 3. The light emission pattern 63 indicates a pattern of timing at which the terminal 3 emits light. For example, “0” means no light emission, and “1” means light emission. When the frame rate is 30 fps, the light emission pattern 63 may include 30 numbers “0” or “1”. The RGB value 62 and the light emission pattern 63 are different combinations for each identification number 61.

  The terminal 3 emits light based on the identification information light emission conversion data 60, the photographing device 6 photographs the display unit of the terminal 3, and the server 2 analyzes the photographing result by the photographing device 6, so that the terminal 3 can communicate with the server 2 itself. Can be transmitted. Further, since the server 2 can also specify the position of the terminal 3 in the pseudo display 4, it becomes possible to map the assigned pixel to each terminal 3.

  Note that the identification information for identifying the terminal 3 is not limited to the identification number 61 of a serial number (for example, a serial number automatically numbered in the association display system 1b) as shown in FIG. The identification number may be used, or the mail address of the user of the terminal 3 may be used.

  FIG. 9 is a flowchart illustrating the flow of the preparatory processing according to the second embodiment. The server 2 and the terminal 3 store the identification information light emission conversion data 60 in the storage unit in advance. As shown in FIG. 9, the terminal 3 downloads an application program and data from the server 2 (Step S301), and installs the application program. This application program is a program for providing a process of steps S302 and S303 described later, that is, a function of transmitting the identification information of the terminal 3 to the server 2 using a color or a light emission pattern. Step S301 may be performed in advance outside the venue. The application includes, in addition to the display processing, the content of a display position specifying processing described later. The data to be downloaded are the moving image data 10 shown in FIG. 2 and the identification information light emission conversion data 60 shown in FIG. 8, and also include the hue sound conversion data 30 and the luminance volume conversion data 40 shown in FIG.

  Next, each terminal 3 converts its own identification information into a light emission pattern based on the identification information light emission conversion data 60 (step S302), and simultaneously displays the light emission pattern on the display unit in the venue (step S303). ). On the other hand, the image capturing device 6 captures a moving image of the display unit of the terminal 3 in the pseudo display 4, and transmits the captured result to the server 2 (step S304).

  Next, the server 2 analyzes a photographing result of the photographing device 6, extracts a light emission pattern for each terminal 3, and specifies a light emission position, an identification number, and a light emission correction value of the terminal 3 (step S305).

  Two methods will be described for the method of specifying the light emitting position of the terminal 3. For example, if the number of seats in the horizontal and vertical directions in the pseudo display 4 is known in advance, the server 2 stores the number of seats in the horizontal and vertical directions in the storage unit as the number of pixels in the horizontal and vertical directions. . Then, the server 2 divides the image of each frame obtained by the imaging device 6 into pixels based on the number of pixels in the horizontal direction and the vertical direction, and specifies the pixel position of the light emitting point of the terminal 3.

  Further, for example, when the numbers of seats in the horizontal and vertical directions in the pseudo display 4 are not known in advance, the server 2 calculates the number of light emitting points from the image obtained by the imaging device 6 and calculates the number of light emitting points according to the aspect ratio of the content. Then, the number of pixels in the horizontal and vertical directions may be calculated from the number of light emitting points. For example, when the number of light emitting points is 1,200 and the aspect ratio of the content is 4: 3, the number of pixels in the horizontal direction is 40 and the number of pixels in the vertical direction is 30. Then, the server 2 divides the image obtained by the imaging device 6 into pixels based on the number of pixels in the horizontal direction and the vertical direction, and specifies the pixel position of the light emitting point of the terminal 3. In this method, each terminal 3 generally handles one pixel. Note that a plurality of terminals 3 may be in charge of the same pixel, or there may be a pixel which is not in charge of any terminal 3. However, in order to eliminate defective pixels in the content, it is desirable to calculate the number of pixels in the horizontal and vertical directions so that one or more terminals 3 are in charge of all the pixels.

  Even if the number of seats in the horizontal and vertical directions in the pseudo display 4 is unknown, the number of pixels in the horizontal and vertical directions may be fixed. That is, the server 2 may store the number of pixels in the horizontal direction and the vertical direction in the storage unit as the set value. In this case, in order to eliminate defective pixels in the content, it is desirable to set the number of pixels in the horizontal and vertical directions small.

  Next, a method of specifying the identification number of the terminal 3 will be described. The server 2 calculates color information and a light emission pattern for each light emitting point from the image obtained by the image capturing device 6. Then, the server 2 searches the identification information light emission conversion data 60 for the closest combination of the RGB value 62 and the light emission pattern 63 and specifies the identification number 61.

  Next, a method of specifying the light emission correction value of the terminal 3 will be described. The display unit of the terminal 3 such as a smartphone has different colors and brightness depending on the model. Therefore, in order to absorb those differences, the server 2 specifies the light emission correction value of the terminal 3 from the image obtained by the imaging device 6. For example, the terminal 3 displays a light emission pattern for each terminal 3 on the display unit, and then displays a light emission pattern for performing color tone correction and luminance correction on the display unit. For example, all the terminals 3 display the light emission pattern with the same luminance setting value, the server 2 calculates the average of the luminance of all the light emission points, and calculates the difference between the luminance of each light emission point and the light emission related to the luminance correction. Set the correction value.

  Returning to the description of FIG. Next, the server 2 transmits the pixel position of the assigned pixel and the light emission correction value for each terminal 3 to the terminal 3 (Step S306). The server 2 also transmits the number of pixels in the horizontal direction and the vertical direction to the terminal 3 as necessary. The server 2 may specify the destination terminal 3 based on the identification number, and transmit the pixel position of the responsible pixel and the light emission correction value via the Internet or one-to-one wireless communication. Alternatively, the server 2 may distribute the associations of the identification numbers of all the terminals 3 with the pixel positions of the assigned pixels and the light emission correction values via WiFi, via WiFi.

  Next, the terminal 3 receives the pixel position and the light emission correction value of the assigned pixel (Step S307), and corrects the output data of the assigned pixel based on the emission correction value (Step S308). For example, the terminal 3 whose luminance at the light emitting point is lower than the average corrects the output data so that the output data of the assigned pixel is displayed with the high luminance set value. Then, the terminal 3 stores the output data of the assigned pixel in the main storage unit (step S309). The terminal 3 decodes the moving image of the content based on the number of pixels in the horizontal direction and the vertical direction received from the server 2 as needed, and extracts output data of the assigned pixel.

  As described above, in the second embodiment, the identification information light emission conversion data 60 including the light emission pattern for each identification number is created in advance, and the terminal 3 displays the light emission pattern corresponding to its own identification number on the display unit. The photographing device 6 photographs the display unit of the terminal 3, and the server 2 analyzes the photographing result of the photographing device 6 to specify the identification number of the terminal 3 and the pixel position of the assigned pixel. Thus, the assigned pixel can be mapped to each terminal 3 even in a venue where the seat number 5 is not defined. Note that the display processing according to the second embodiment can be executed in the same manner as the display processing according to the first embodiment, and a description thereof will be omitted.

  FIG. 10 is a diagram illustrating an example of the input information light emission conversion data 70. The input information light emission conversion data 70 is stored in the storage units of the server 2 and the terminal 3 in advance. In the second embodiment, predetermined input information light emission conversion data 70 is used to total the questionnaire results for the audience.

  The input information light emission conversion data 70 includes, for example, an RGB value 72 and a light emission pattern 73 for each input value 71 relating to a questionnaire response. The RGB value 72 is a data item having the same meaning as the RGB value 62 shown in FIG. The light emission pattern 73 is a data item having the same meaning as the light emission pattern 63 shown in FIG. The RGB value 72 and the light emission pattern 73 are different combinations for each input value 71. In the example shown in FIG. 10, the questionnaire is answered by a number such as 1, 2, 3,.

  FIG. 11 is a flowchart showing the flow of the questionnaire totalizing process in the second embodiment. In the second embodiment, the server 2 can execute an information collection process of collecting information from a plurality of terminals 3 at a time. The questionnaire totalizing process is an example of an information collecting process.

  The server 2 and the terminal 3 store the input information light emission conversion data 70 in the storage unit in advance. As shown in FIG. 11, the terminal 3 receives a questionnaire result from a user and inputs the questionnaire result via the input unit (Step S401).

  Next, the terminal 3 converts the questionnaire result input in step S401 into a light emission pattern based on the input information light emission conversion data 70 (step S402), and simultaneously displays the converted light emission pattern in the hall. (Step S403). On the other hand, the photographing device 6 photographs the display unit of the terminal 3 in the pseudo display 4 and transmits the photographed result to the server 2 (step S404).

  Next, the server 2 analyzes the photographing result of the photographing device 6, extracts a light emission pattern for each terminal 3, and specifies a questionnaire result (step S405). Specifically, the server 2 calculates the color information and the light emission pattern for each light emitting point from the image obtained by the image capturing device 6. Then, the server 2 searches the input information light emission conversion data 70 for the closest combination of the RGB value 72 and the light emission pattern 73 and specifies the input value 71.

  Then, the server 2 totals the questionnaire results (step S406). The server 2 may transmit the questionnaire result to the terminal 3 via BLE or the like. When a large screen display different from the pseudo display 4 exists in the venue, the server 2 may output the questionnaire results totaled in step S406 to the large screen display.

  The number of the terminals 3 is not limited as long as the light emission of the display unit of each terminal 3 can be identified by the photographing result of the photographing device 6. Since the server 2 can simultaneously read the questionnaire results from the terminal 3 in the pseudo display 4, the server 2 can also collect the questionnaire results of tens of thousands of people at a high speed. Thereby, a moving image to be reproduced can be selected based on the questionnaire result. For example, video content with a large number of voting results can be preferentially reproduced. In addition, the questionnaire results can be used for effects such as gender, age, and a display that distinguishes who the fan is. For example, the display content can be changed for each group of the seats (for example, the left side and the right side) based on information such as gender.

  The cooperative display system 1b can also implement a message collection process as an example of the information collection process. For example, the input information light-emission conversion data 70 shown in FIG. 10 is extended, and light-emitting patterns are associated not only with numbers but also with characters (for example, light-emitting patterns are defined in association with character codes). The terminal 3 switches the light emission pattern for each character constituting the message to be sent and displays the light emission pattern on the display unit. The server 2 analyzes the photographing result of the photographing device 6, specifies a character for each light emission pattern, and restores a message. Then, the server 2 may output the restored message on a large screen display different from the pseudo display 4 in real time. Thereby, for example, the output is started to the large-screen display every time the message is restored, and a display form in which a plurality of messages flow like a river in chronological order can be realized.

  As described above, the preferred embodiments of the cooperative display system and the like according to the present invention have been described with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious to those skilled in the art that various changes or modifications can be made within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. I understand.

1a, 1b ... Display coordination system 2 ... Server 3 ... Terminal 4 ... Pseudo display 5 ......... seat number 6 ......... shooting device 10 ......... video data 20 ......... map data 30 .. ....... Hue sound conversion data 40 ...... Brightness / volume conversion data 50 ......... Output data 60 ......... Identification information emission Conversion data 70 ............ Input information light emission conversion data

Claims (7)

A cooperative display system in which a plurality of terminals having a display unit cooperate to constitute a single pseudo display, and display moving image content,
A server capable of communicating with the terminal,
The terminal stores moving image data including display information of all pixels of the content,
Wherein the terminal accepts an input of a seat number;
The terminal, based on the map data indicating the association between the seat number and the pixel position in the pseudo display, identifying the pixel position of the pixel that is responsible for itself,
The terminal, from the video data, extracting the display information regarding the specified pixel position,
Advance preparation processing including
The terminal displaying the display information according to a command from the server;
A display process including
And a cooperative display system. The advance preparation process further includes:
A step in which the terminal is in a state where the display information to be extracted can be immediately read in chronological order;
The link display system according to claim 1, further comprising: The advance preparation process further includes:
Wherein the terminal specifies an audio channel and a volume based on information obtained from the display information;
The terminal, the sound channel based on the specified audio channel and the sound volume in a state that can be immediately read in chronological order sound information,
The link display system according to claim 2, comprising: The display processing further includes:
The server transmits a display start command of a first frame of the moving image data to the terminal,
The terminal, when receiving the display start command, displaying the display information in chronological order from the first frame,
The server transmitting a display synchronization command to the terminal at predetermined intervals;
If the terminal displays the display information for a predetermined number of frames before receiving the display synchronization command, the terminal displays the display information for the next frame for the predetermined number of frames after receiving the display synchronization command, When the display synchronization command is received before displaying the display information for a predetermined number of frames, displaying the display information on the next frame for the predetermined number of frames immediately,
The linked display system according to any one of claims 1 to 3, further comprising:   4. The cooperative display system according to claim 3, wherein in the display processing, the terminal outputs the sound information in synchronization with displaying the display information. A cooperative display system in which a plurality of terminals having a display unit cooperate to constitute a single pseudo display, and display moving image content,
A server capable of communicating with the terminal, and a photographing device that photographs an area in the pseudo display,
The terminal, a step of storing a program for providing a function of transmitting the identification information of the terminal to the server using a color or light emission pattern,
The terminal, based on the identification information light emission conversion data indicating the correspondence between the identification information and the color or the light emitting pattern of the terminal, the step of converting its own identification information into the color or the light emitting pattern,
The terminal displays the color or the light emission pattern to be converted,
The image capturing apparatus captures the display unit of the terminal included in the pseudo display,
The server analyzes a photographing result by the photographing device, and extracts the color or the light emission pattern for each terminal;
The server specifying a pixel position in the pseudo display based on the position of the color or the light emission pattern to be extracted;
The server refers to the identification information light emission conversion data, and specifies the identification information based on the extracted color or the light emission pattern,
The server transmits the pixel position associated with the identification information to the terminal,
The terminal receives the pixel position associated with the identification information, from the content, extracting display information about the pixel position associated with its own identification information,
Preparation process including
The terminal displaying the display information according to a command from the server;
A display process including
And a cooperative display system. A cooperative display system in which a plurality of terminals having a display unit cooperate to constitute a single pseudo display, and display moving image content,
A server capable of communicating with the terminal, and a photographing device that photographs an area in the pseudo display,
The terminal inputs information input by the user,
The terminal, based on input information light emission conversion data indicating the correspondence between the input information and color or light emission pattern, the input information to be input is converted to the color or the light emission pattern,
The terminal displays the color or the light emission pattern to be converted,
The image capturing apparatus captures the display unit of the terminal included in the pseudo display,
The server analyzes a photographing result by the photographing device, and extracts the color or the light emission pattern for each terminal;
A step in which the server refers to the input information light emission conversion data and specifies the input information based on the extracted color or the light emission pattern;
A cooperative display system characterized by executing an information collection process including:

Images