WO2023062802A1

WO2023062802A1 - Content distribution method and content distribution system

Info

Publication number: WO2023062802A1
Application number: PCT/JP2021/038157
Authority: WO
Inventors: 悠二米原; 帝聡黒木; 由希子浅野; 久幸三木; 大倫佐藤
Original assignee: 豊田合成株式会社
Priority date: 2021-10-15
Filing date: 2021-10-15
Publication date: 2023-04-20

Abstract

This content distribution method includes a first acquisition step, a second acquisition step, a determination step, and a distribution step. The first acquisition step includes acquiring a video and sound information by filming a performer. The second acquisition step includes acquiring tactile information of the performer. The determination step includes determining whether or not the video acquired in the first acquisition step contains a determination matter related to switching of the tactile information to be distributed. The distribution step includes distributing acquired content information. The distribution step includes a first distribution step for deciding and distributing the video and the sound information to be distributed and a second distribution step for distributing the tactile information corresponding to the video decided in the first distribution step. When the video decided in the first distribution step contains the determination matter, distribution of the tactile information is started in the second distribution step after a predetermined delay time has passed from the start of distribution of the video and the sound information in the first distribution step.

Description

CONTENT DISTRIBUTION METHOD AND CONTENT DISTRIBUTION SYSTEM

The present invention relates to a content distribution method and a content distribution system.

A technique is known for increasing the sense of immersion and realism in a live broadcast of a user who watches the live broadcast of a concert. For example, Patent Literature 1 discloses a technique for distributing tactile information based on biometric information of a performer superimposed on video and sound information of a concert to a user via a wearable device worn by the user. The actor's biological information is, for example, the heartbeat. In this case, a user watching the live distribution is presented with a tactile sensation based on the performer's biometric information in addition to the visual information and auditory information. This enhances the user's sense of immersion and presence in the live distribution.

JP 2016-197350 A

　Conventional content distribution methods have room for improvement in terms of improving the sense of immersion and realism in live distribution.

A content distribution method for solving the above problems is a content distribution method for live distribution of content information including a video of an actor, sound information, and tactile information for operating a tactile presentation device, wherein the actor is photographed. a first acquisition step of acquiring the video and the sound information by means of a second acquisition step of acquiring or generating the tactile information of the performer; and a delivery step of delivering the acquired content information, wherein the delivery step includes the video and the a first distribution step of determining and distributing the sound information; and a second distribution step of distributing the tactile information corresponding to the video determined in the first distribution step, wherein If the video determined by the above includes the determination item, in the second distribution step, after a predetermined delay time has elapsed from the start of distribution of the video and the sound information in the first distribution step, Delivery of the tactile information is started.

According to the above configuration, when the tactile information distributed in the second distribution step is switched, a delay time is provided before the tactile information after switching is distributed. In other words, the delivery of tactile information after switching is delayed. During this delay time, the tactile sensation presented to the user from the tactile sensation presentation device is adjusted to control the operation of the tactile sensation presentation device during the delay time, thereby controlling the impression of the tactile sensation based on the tactile signal delivered after the delay time. can. As a result, it is possible to expand the range of expression of tactile sensations based on the tactile sensation signals distributed after the delay time. In addition, it is possible to effectively improve the user's sense of immersion and presence in the live distribution.

According to one aspect of the content distribution method, in the delay time, modulated tactile information obtained by modulating the tactile information distributed before the delay time, after the delay time, or before and after the delay time, or modulated tactile information distributed in advance. Distributing connecting tactile information for connection, and switching from distribution of the modulated tactile information or the connecting tactile information to distribution of the tactile information after the delay time.

In one aspect of the content distribution method, the first obtaining step is a step of obtaining a plurality of videos at the same time, and the content distribution method performs at least a part of the processing in the determination step in the distribution step. simultaneously for each of the images.

In one aspect of the content distribution method, there are a plurality of performers, and the determination item is that the performer appearing in the video is different from the performer from which the currently distributed tactile information is derived. be.

A content distribution system that solves the above problems includes a photographing device that acquires the video and the sound information, a tactile sensation acquisition device that acquires or generates the tactile information of the performer, and the distributed video and the sound information. a viewing device configured to be used by a user for viewing; a tactile presentation device configured to present to the user a tactile sensation based on the distributed tactile information; an information processing device for delivering to a viewing device and the tactile presentation device, wherein the information processing device includes a determination unit that determines whether or not the image captured by the imaging device includes the determination item; a first distribution control unit that determines and distributes the video and the sound information to be distributed; and a second distribution control unit that distributes the tactile information corresponding to the video determined by the first distribution control unit. When the video determined by the first distribution control unit includes the determination item, the second distribution control unit controls distribution of the video and the sound information by the first distribution control unit. After a predetermined delay time has elapsed from the start, distribution of the tactile information is started.

1 is a block diagram of a content distribution system; FIG. It is explanatory drawing of a tactile sense presentation apparatus. 1 is a schematic cross-sectional view of a tactile sensation presentation device; FIG. 4 is a flow chart of a first pattern and a second pattern of a content distribution method; 4 is a timing chart in the first pattern of the content distribution method; It is a timing chart in the second pattern of the content delivery method. It is a timing chart in the 3rd pattern of the content delivery method. FIG. 11 is a flow chart of a third pattern of a content distribution method; FIG.

An embodiment of a content distribution system for live distribution of content information including video, sound information, and tactile information of performers will be described below.
<Content distribution system>
As shown in FIG. 1, the content distribution system 10 includes a photographing device 11, a tactile sensation acquisition device 12, a viewing device 13, a tactile sensation presentation device 14, and an information processing device 15 that distributes content information.

(imaging device)
The photographing device 11 is not particularly limited, and a known photographing device such as a camera for acquiring sound information such as images and voices of performers can be applied. The image capturing device 11 included in the content distribution system 10 may be singular or plural. The image and sound information acquired by the photographing device 11 is transmitted to the information processing device 15 .

(tactile sensation acquisition device)
The tactile sense acquisition device 12 is a device for acquiring the tactile sense information of the actor. The tactile information acquired by the tactile acquisition device 12 includes, for example, vibrations generated based on the actions of the performer, and biological information such as pulse, heartbeat, respiration, and blood pressure. As the tactile sensation acquisition device 12, a known measurement device suitable for tactile information to be acquired can be applied. The tactile sense acquisition device 12 may be a contact type device worn by the performer, or may be a non-contact type device.

Examples of the tactile sensation acquisition device 12 include a pressure sensor, pulse meter, heart rate meter, sphygmomanometer, and thermometer. Further, the tactile sensation acquisition device 12 may be a device that acquires tactile information by analyzing the image of the performer captured by the imaging device 11 . The content distribution system 10 may include a single tactile sensation acquisition device 12 or a plurality of tactile sensation acquisition devices. When there are a plurality of performers, a tactile sensation acquisition device 12 is provided for each performer. Also, a plurality of tactile sensation acquisition devices 12 may be provided for one performer. The tactile information acquired by the tactile acquisition device 12 is transmitted to the information processing device 15 .

(viewing device)
The viewing device 13 is a display device having an audio output function for allowing the user to view the actor's video and audio information distributed from the information processing device 15 . That is, the viewing device 13 is used by the user to view the actor's video and sound information distributed from the information processing device 15 . The viewing device 13 includes a display device having an audio output function, for example, a tablet terminal or mobile terminal display, a stationary display, or a head-mounted display device such as a head-mounted display. Also, the viewing device 13 may be a combination of a display device without an audio output function and an audio output device.

(tactile sensation presentation device)
The tactile sensation presentation device 14 is a device for presenting a tactile sensation to the user watching the live distribution. The tactile sensation presentation device 14 presents a tactile sensation based on the tactile information to the user by vibrating based on the tactile information distributed from the information processing device 15 . An example of the tactile presentation device 14 is shown in FIGS. 2 and 3. FIG.

The tactile sensation presentation device 14 shown in FIGS. 2 and 3 is a device that is used while being held in one hand by the user. The tactile sensation presentation device 14 includes a box-shaped housing 20 sized to fit in the user's hand, and a presentation unit 21 provided on the upper surface of the housing 20 . The presentation unit 21 includes a soft layer 22 protruding in a hemispherical shape from the upper surface of the housing 20 and one or more vibration actuators 23 arranged on the upper surface of the soft layer 22 . The soft layer 22 is made of a soft material such as urethane. The vibration actuator 23 is a sheet-like dielectric elastomer actuator (DEA: Dielectric Elastomer Actuator).

The DEA as the vibration actuator 23 causes the user who touches the upper surface of the presentation unit 21 to perceive vibrations based on deformation such as expansion and contraction of the DEA as a specific tactile sensation. The soft layer 22 is a layer that deforms following deformation of the DEA. By arranging the soft layer 22 between the DEA and the housing 20 , it is possible to suppress deformation of the DEA from being restricted by the housing 20 .

A DEA is a multi-layer structure in which a sheet-like dielectric layer made of a dielectric elastomer and a plurality of positive electrodes and negative electrodes as electrode layers arranged on both sides of the dielectric layer in the thickness direction are laminated. An insulating layer is laminated on the outermost layer of the DEA. In DEA, when a DC voltage is applied between the positive electrode and the negative electrode, the dielectric layer is compressed in the thickness direction and along the surface of the dielectric layer depending on the magnitude of the applied voltage. It deforms so as to extend in the plane direction of the DEA. The DEA allows the user to perceive vibrations and the like based on the expansion and contraction of the DEA as a tactile sensation.

The dielectric elastomer constituting the dielectric layer is not particularly limited, and known dielectric elastomers used in DEA can be used. Examples of the dielectric elastomer include crosslinked polyrotaxane, silicone elastomer, acrylic elastomer, and urethane elastomer. One type of these dielectric elastomers may be used, or a plurality of types may be used in combination. The thickness of the dielectric layer is, for example, 20-200 μm.

Examples of materials that make up the positive and negative electrodes include conductive elastomers, carbon nanotubes, Ketjenblack (registered trademark), and vapor-deposited metal films. Examples of the conductive elastomer include a conductive elastomer containing an insulating polymer and a conductive filler.

Examples of the insulating polymer include crosslinked polyrotaxane, silicone elastomer, acrylic elastomer, and urethane elastomer. One type of these insulating polymers may be used, or a plurality of types may be used in combination. Examples of the conductive filler include carbon nanotubes, Ketjenblack (registered trademark), carbon black, and metal particles such as copper and silver. One type of these conductive fillers may be used, or a plurality of types may be used in combination. The thickness of the positive electrode and negative electrode is, for example, 1 to 100 μm.

The insulating elastomer constituting the insulating layer is not particularly limited, and known insulating elastomers used for the insulating portion of known DEA can be used. Examples of the insulating elastomer include crosslinked polyrotaxane, silicone elastomer, acrylic elastomer, and urethane elastomer. One type of these insulating elastomers may be used, or a plurality of types may be used in combination. The thickness of the insulating layer is, for example, 10-100 μm. Moreover, the thickness of the entire DEA is preferably, for example, 0.3 to 3 mm from the viewpoint of ensuring flexibility and strength.

A coating layer (not shown) that covers the vibration actuator 23 is arranged on the vibration actuator 23 as necessary. Further, inside the housing 20 of the tactile sense presentation device 14, a driving unit 24 for applying a voltage between a pair of electrodes constituted by a positive electrode and a negative electrode of the vibration actuator 23 from a power supply (not shown) such as a battery is provided. is provided.

Note that the tactile sensation presentation device 14 is not limited to the tactile sensation presentation device 14 shown in FIGS. For example, it may be a penlight-type tactile sensation presentation device 14 in which a vibration actuator 23 is arranged in the grip portion of a penlight used at a concert or the like, or it can be used by holding it like a cushion or pillow. The tactile presentation device 14 may be a hugging type tactile presentation device 14 in which the vibration actuator 23 is arranged on a certain cushion material. The penlight-type tactile sensation presentation device 14 is suitable for enhancing the presence of live distribution. In addition, the hugging type tactile sense presentation device 14 is suitable for viewing live distribution in a relaxed state.

(Information processing device)
The information processing device 15 sets content information to be distributed to the viewing device 13 and the tactile presentation device 14 based on the acquired video, sound information, and tactile information. In addition, the information processing device 15 sets the set content information, that is, the actor's video, sound information, and tactile information so that the viewing device 13 can view and the tactile presentation device 14 can present the tactile sensation. deliver to Examples of the information processing device 15 include a server, a PC (Personal Computer), a mobile phone such as a smart phone, a tablet terminal, and a game machine.

As shown in FIG. 1 , the information processing device 15 includes a transmission/reception section 30 , a signal generation section 31 , a determination section 32 , a first distribution control section 33 , a second distribution control section 34 and a storage section 35 .
The transmission/reception unit 30 is, for example, a communication interface for communicating with the photographing device 11, the tactile sensation acquisition device 12, the viewing device 13, and the tactile sensation presentation device 14 via the Internet as a wide area communication network. For example, the transmission/reception unit 30 may be compatible with a mobile communication system so as to be directly connected to the Internet, or may communicate with devices connected to the Internet. The wide area communication network is arbitrary, and may be, for example, a telephone line. Further, transmission and reception between the transmission/reception unit 30 and the imaging device 11 and the tactile sensation acquisition device 12 may be performed by wire.

The signal generation unit 31, the determination unit 32, the first distribution control unit 33, and the second distribution control unit 34 include: 1) one or more processors that operate according to a computer program (software); or 3) any combination thereof, such as one or more dedicated hardware circuits, such as an application specific integrated circuit (ASIC), that performs the processing of . A processor includes, for example, a CPU. Each process executed by the signal generation unit 31, the determination unit 32, the first distribution control unit 33, and the second distribution control unit 34 described below is executed using a control program stored in the storage unit 35. be.

The signal generation unit 31 generates a tactile signal from the tactile information acquired by the tactile acquisition device 12 . The tactile sensation signal is a signal for driving the vibration actuator 23 of the tactile sensation presentation device 14 so as to present a tactile sensation based on the acquired tactile information. When there are a plurality of tactile sensation acquisition devices 12 , the signal generation unit 31 generates a tactile signal from the tactile information acquired for each tactile sensation acquisition device 12 . In other words, the corresponding tactile signals are generated for all the tactile information acquired by the tactile sense acquisition device 12 . The signal generation unit 31 also generates a modulated tactile signal by modulating the generated tactile signal. The details of the modulated tactile signal will be described later. Each tactile signal described above is a signal indicating a voltage waveform, and the volume of the tactile signal is the amplitude of the voltage waveform.

The determination unit 32 determines whether or not the pre-distribution video acquired by the imaging device 11 includes preset determination items. The determination item is a requirement set in advance for determining switching of tactile information to be distributed. The determination item of the present embodiment is that the main performer appearing in the video determined to be distributed is different from the performer from which the tactile information currently distributed is derived. If the main performer appearing in the video determined to be distributed is different from the performer from which the tactile information currently distributed is derived, the determination unit 32 makes an affirmative determination. Then, when the main performer appearing in the video decided to be distributed is the same as the performer from which the tactile information currently distributed is derived, the determination unit 32 makes a negative determination. It should be noted that at a timing when there is no tactile information being distributed at present, such as immediately after the start of distribution, the determination unit 32 makes a negative determination. Examples of the determination method by the determination unit 32 include a known determination method such as a determination method using AI for image recognition. Also, the timing of the determination by the determination unit 32 can be arbitrarily set during the period before the video used for determination is distributed.

The first distribution control unit 33 determines video and audio information to be distributed from the video and audio information acquired by the photographing device 11, and distributes the determined video and audio information so that it can be viewed by the viewing device 13. . When there is a single imaging device 11, the first distribution control unit 33 determines video and audio information acquired by one imaging device 11 as video and audio information to be distributed. Then, the first distribution control unit 33 performs control for distributing the determined video and audio information. The first distribution control unit 33 distributes the image and sound information with a slight time difference between the acquisition of the image and sound information by the photographing device 11 and the distribution. This time difference is, for example, several seconds to ten minutes. The method of providing this time difference is not particularly limited, and an existing distribution technology such as delay broadcasting may be used. At the timing based on the time difference from when the image and sound information is acquired by the photographing device 11 to when the information is distributed, the determination process by the determination unit 32 for the image immediately before distribution is performed.

When there are a plurality of photographing devices 11, the first distribution control unit 33 determines the video to be distributed from the plurality of videos acquired by the plurality of photographing devices 11. Also, the first distribution control unit 33 determines sound information to be distributed from a plurality of pieces of sound information acquired by a plurality of photographing devices 11 . Then, the first distribution control unit 33 distributes the determined video and audio information while providing the time difference. The process of determining the video to be distributed from a plurality of videos and the process of determining the sound information to be distributed from a plurality of sound information are performed, for example, based on the switching operation performed by the operator to switch the content information to be distributed. .

The second distribution control unit 34 distributes the tactile signal generated from the tactile information so that the tactile sensation presentation device 14 can present the tactile sensation. The tactile signal distributed by the second distribution control unit 34 is a tactile signal based on the main performer appearing in the video determined to be distributed by the first distribution control unit 33 .

The second distribution control unit 34 performs delay control to delay the start of distribution of the tactile signal when the determination by the determination unit 32 for the video determined to be distributed by the first distribution control unit 33 is positive. The delay control is control for starting distribution of the tactile signal after a predetermined delay time has elapsed since the first distribution control unit started distributing the video and audio information. The delay time is, for example, 100 milliseconds to 3 seconds. If the video to be distributed has a transition effect, the delay time is preferably set to match the length of the transition effect.

The delay time is provided to expand the range of tactile sensations presented to the user based on the tactile signals delivered after the delay time. Whether or not to distribute a signal for operating the tactile sensation presentation device 14 during the delay time can be set arbitrarily. By controlling the operation of the tactile sense presentation device 14 during the delay time, it is possible to control the impression of the tactile sense presented to the user based on the tactile sense signal delivered after the delay time.

For example, the delay time is the time during which no tactile signal is distributed so as not to operate the tactile sensation presentation device 14 . In this case, the tactile sensation presented to the user from the tactile sensation presentation device 14 after the delay time is emphasized. In addition, it is possible to emphasize that the tactile sensation presented to the user from the tactile sensation presentation device 14 has been switched. The delay time is defined as the time for delivering the connecting tactile information for operating the tactile sensation presentation device 14 so as to present the connecting tactile sensation to the user. In this case, it is possible to reduce the discomfort that the user experiences due to the sudden change in the tactile sensation presented to the user by the tactile sensation presentation device 14 . As the tactile information for connection, for example, modulated tactile information obtained by modulating the tactile information distributed before the delay time, after the delay time, or before and after the delay time, and high affinity with various tactile sensations. tactile information for connection created in advance as shown in FIG.

In this embodiment, as an example, a specific description is given of a case in which the delay time is the time to distribute the modulated tactile information for operating the tactile sensation presentation device 14 so as to present a connecting tactile sensation to the user. ing. The second distribution control unit 34 distributes the modulated tactile signal as the modulated tactile information during the delay time so that the tactile sensation presentation device 14 can present the tactile sensation. A modulated haptic signal is a signal obtained by superimposing a fade-out signal and a fade-in signal. The fade-out signal is a signal obtained by modulating the haptic signal distributed before the delay time so that it gradually becomes smaller during the delay time. The fade-in signal is a signal obtained by modulating the haptic signal delivered after the delay time so that it gradually increases during the delay time. After the delay time has elapsed, the second distribution control unit 34 switches from distribution of the modulated tactile signal to distribution of the tactile signal.

A control program is stored in the storage unit 35 . The storage unit 35 is, for example, a nonvolatile memory.
<Content delivery method>
Next, a content distribution method for live-distributing content information including video, sound information, and tactile information of actors using the content distribution system 10 described above will be described.

(first pattern)
As shown in the timing chart of FIG. 5, the first pattern of the content distribution method is a content distribution method in which there are two performers A and B, and the photographing device 11 is _one camera CA. is.

Based on the flowchart of FIG. 4, the following cases will be described as examples. As shown in FIG. 5, for a while after the start of distribution, actor A is photographed by camera _CA and content information of actor A is distributed. After that, the actor B is photographed by the camera _CA , and the distribution of the content information of the actor A is switched to the distribution of the content information of the actor B.

Step S1 is a first obtaining step of photographing performers A and B with a camera _CA to obtain video and sound information. In step S1, the cameraman takes an image of performer _A with camera CA, and after a predetermined time has elapsed, changes the object of photography from performer A to performer B, and then takes performer B. While the actor _A is being photographed by the camera CA, video and audio information based on the actor A is transmitted from the camera _CA to the information processing device 15 . Further, while camera _CA is photographing performer B, video and sound information based on performer B is transmitted from camera _CA to information processing device 15 .

Step S2 is a second acquisition step in which the tactile information of performer A and the tactile information of performer B are acquired by the tactile acquisition device 12 respectively. Acquisition of each tactile information of actor A and actor B by the tactile sensation acquisition device 12 is performed continuously or intermittently throughout the live distribution. Then, each acquired tactile information is transmitted from the tactile sensation acquisition device 12 to the information processing device 15 . Note that steps S1 and S2 are executed in parallel.

Step S3 is a step of determining the video and audio information to be distributed acquired in step S1. In this pattern in which one camera is camera _CA , one type of video and audio information is distributed. Therefore, in step S3, the first distribution control unit 33 always determines the video and sound information corresponding to the video acquired by one camera _CA as the video and audio information to be distributed.

Step S4 is a judgment step for judging whether or not the pre-distribution video whose distribution was decided in step S3 contains a judgment item. In step S3, the determination unit 32 determines that the main performer appearing in the video before distribution, which has been determined to be distributed in step S3, is currently being distributed. A positive determination (YES) or a negative determination (NO) is made based on whether the performer is different from the information source. At the start of distribution, since there is no tactile information being distributed at present, the determination by the determination unit 32 is a negative determination.

Steps S5a and S5b are steps for starting distribution of the determined video and audio information. Step S5a is a step performed when the determination in step S4 is negative, and step S5b is a step performed when the determination in step S4 is positive. Steps S5a and S5b differ in that the process is performed when the determination in step S4 is affirmative or the process is performed when the determination in step S4 is negative. are the same.

In steps S5a and S5b, the first distribution control unit 33 starts distributing the video and audio information with a slight time lag between the acquisition of the video and audio information by the photographing device 11 and the distribution thereof.

Step S<b>6 is a step of distributing the tactile signal acquired by the tactile sensation acquisition device 12 . In step S6 during the period in which camera C _A continues to photograph actor A, if actor A is the main actor appearing in the images distributed in steps S4a and S4b, second distribution control unit 34: A tactile signal based on the tactile information of performer A is distributed in synchronization with the video. Then, when the main performer is performer B, the second distribution control unit 34 distributes the tactile signal based on the tactile information of performer B in synchronization with the video.

Step S7 is a step of distributing a modulated tactile signal as modulated tactile information for a predetermined delay time. Step S7 is performed following step S5a when the determination in step S4 is affirmative. As shown in FIG. 4, the second distribution control unit 34 distributes a modulated tactile signal obtained by superimposing a fade-out signal and a fade-in signal. A fade-out signal is a signal obtained by modulating a tactile signal based on tactile information of an actor, for example, actor A, which has been distributed before the delay time. The fade-in signal is a signal obtained by modulating a tactile signal based on the tactile information of an actor, for example, actor B appearing in the video distributed in step S5a.

If the determination in step S4 is negative, the process moves from step S4 to steps S5a and S6. If the determination in step S4 is affirmative, the process proceeds from step S4 to step S5b and step S7, and then from step S7 to step S6. After step S6, the information processing device 15 returns to step S4, and repeats the processes after step S4. Accordingly, the information processing device 15 continuously distributes video, sound information, and tactile information.

As shown in FIG. 5, for a while after the start of the distribution, the actor A continues to be photographed by the camera _CA. At this time, video and audio information based on actor A is transmitted from camera _CA to information processing device 15 . During this period, the actor appearing in the video determined to be distributed in step S3 is actor A, and in step S6, actor A is also the actor from whom the tactile information currently being distributed is derived. be.

Therefore, the determination of the determination unit 32 in step S4 during this period is a negative determination, including at the start of distribution. If the determination in step S4 is a negative determination, the determination by the determination unit 32 is a negative determination because there is no tactile information being distributed at the start of distribution. At this time, in step S5a, the first distribution control unit 33 starts distributing the image decided to be distributed in step S3, that is, the image of actor A and the corresponding sound information. Then, in step S6, the second distribution control unit 34 distributes a tactile signal based on the tactile information of performer A whose video is distributed in step S5a.

In step S1 after a predetermined period of time has elapsed from the start of distribution, the subject of photography is changed from performer A to performer B by the cameraman. As a result, video and audio information based on performer B is transmitted from camera _CA to information processing device 15 .

In step S4 in the cycle immediately after the shooting target is changed from actor A to actor B, actor B is the actor appearing in the video determined to be distributed in step S3. Actor A is the actor from whom tactile information that is currently distributed is derived. In this case, since the two performers are different, the determination of the determination unit 32 in step S4 is a positive determination. At this time, in step S5b, the first distribution control unit 33 starts distributing the image decided to be distributed in step S3, that is, the image of actor B and the corresponding sound information. Then, in step S7, the second distribution control unit 34 distributes the modulated tactile signal for a predetermined delay time from the start of distribution of the video of actor B, and then in step S6, the video is distributed in step S5b. A tactile signal based on the tactile information of performer B who is performing is distributed. During the delay time, the first distribution control unit 33 continues distributing the image of actor B, which is the image obtained in step S1.

After that, the camera _CA continues to photograph the performer B. During the period in which the shooting of actor B is continued, the actor appearing in the video determined to be distributed in step S3 is actor B, and in step S6, the origin of the currently distributed tactile information is determined. Actor B is also Actor B. Therefore, the determination by the determination unit 32 in step S4 in this period is a negative determination. After step S4, distribution is performed in steps S5a and S6.

Also, the user can use the viewing device 13 to view the video and audio information distributed in steps S5a and S5b. At the same time, the vibration actuator 23 of the tactile sense presentation device 14 vibrates based on the tactile sense signal delivered in step S6 and the modulated tactile sense signal delivered in step S7. As a result, the tactile sensation based on actor A or actor B is presented to the user touching the presentation unit 21 of the tactile sensation presentation device 14 . In the first pattern, the first distribution steps are step S3 and steps S5a and S5b. The second distribution steps are steps S6 and S7.

(second pattern)
As shown in the timing chart of FIG. 6, in the second pattern of the content distribution method, there are two performers A and B, and the photographing device 11 captures the performer A with the camera C _A and the performer B. This is a content distribution method in the case of two cameras _CB for photographing .

Based on the flowchart in FIG. 4, the following case will be described as an example. As shown in FIG. 6, the content information of performer A is distributed for a while after the start of distribution. After that, the distribution of the content information of the performer A is switched to the distribution of the content information of the performer B.

In step S1, acquisition of image and sound information based on performer _A by camera CA and acquisition of image and sound information based on performer _B by camera CB are performed simultaneously. The acquired video and sound information based on the performer A is transmitted from the camera _CA to the information processing device 15 . The acquired video and sound information based on actor B is transmitted from camera _CB to information processing device 15 .

Step S2 is similar to the first pattern.
Step S3 is a step of determining the video and audio information to be distributed acquired in step S1. In this pattern in which there are two cameras, camera _CA and camera _CB , there are two types of video and audio information to be distributed. Therefore, in step S3, the first distribution control unit 33 performs distribution based on the video and sound information based on the actor A acquired by the camera C _A and the video and sound information based on the actor _B acquired by the camera CB. determine the video and audio information to be used. The above determination is made based on the input of the switching operation performed by the operator in order to switch the content information to be distributed.

Steps S4 to 7 are the same as in the first pattern.
As shown in FIG. 6, the content information of performer A is distributed for a while after the start of distribution. In step S3 of this period, the first distribution control unit 33 determines the video and sound information based on the performer A acquired by the camera _CA as the video and sound information to be distributed. During this period, the actor appearing in the video determined to be distributed in step S3 is actor A, and in step S6, actor A is also the actor from whom the tactile information currently being distributed is derived. be. The processing after step S4 in this period is the same as in the first pattern. That is, in this period, the determination of the determination unit 32 in step S4 is a negative determination. After that, as step S5a, distribution of the image decided to be distributed in step S3, that is, the image of actor A and the corresponding sound information is started. Then, in step S6, the second distribution control unit 34 distributes a tactile signal based on the tactile information of performer A whose video is distributed in step S5a.

As shown in FIG. 6, the content information of performer B is distributed for a while after a predetermined time has elapsed from the start of distribution. In step S3 of this period, the first distribution control unit 33 determines the video and sound information based on actor B obtained by camera _CB as the video and sound information to be distributed. That is, in step S3 after a predetermined time has elapsed from the start of distribution, the first distribution control unit 33 performs distribution so as to switch from the video and sound information based on performer A to the video and sound information based on performer B. Determine video and audio information.

In step S4 of the cycle immediately after the video and sound information determined in step S3 is changed to the video and sound information based on actor B, the actor appearing in the video determined to be distributed in step S3 is actor B. Actor A is the actor from whom tactile information that is currently distributed is derived. In this case, since the two performers are different, the determination of the determination unit 32 in step S4 is a positive determination. If the determination in step S4 is affirmative, the process moves from step S4 to step S5b and step S7, and after step S7 moves to step S6.

Thereafter, during the period in which the content information of performer B is distributed, the first distribution control unit 33 determines the image and sound information based on performer B acquired by camera _CB as the image and sound information to be distributed. continue. During this period, the actor appearing in the video determined to be distributed in step S3 is actor B, and in step S6, actor B is also the actor from whom the tactile information currently being distributed is derived. be. Therefore, the determination of the determination unit 32 in step S4 in this period is negative including the time of starting the distribution. After step S4, distribution is performed in steps S5a and S6. Specific processing in steps S4 to S7 is the same as in the first pattern.

(Third pattern)
As shown in the timing chart of FIG. 7, in the third pattern of the content distribution method, there are two performers A and B, and the photographing device 11 captures the performer A with the camera C _A and the performer B. This is a content distribution method in the case of two cameras _CB for photographing .

A third pattern of the content distribution method will be described based on the flowchart shown in FIG. The third pattern differs from the second pattern in the configuration of the determination step. Regarding the third pattern, the description of the parts common to the second pattern will be omitted.

The content distribution method of the third pattern has steps S4a and S4b as determination steps instead of step S3 of the second pattern.
Step S4a is a step of determining whether or not each image acquired in step S1 includes a determination item, and storing the determination result. In step S4a, the determination unit 32 intermittently determines whether or not each image acquired in step S1 includes a determination item at predetermined intervals after acquisition. Then, the determination unit 32 stores the determination result in the storage unit 35 for each image acquired in step S<b>1 or updates the determination result stored in the storage unit 35 .

Step S4b is a step for judging whether or not the video decided to be distributed in step S3 contains judgment items based on the most recent judgment results stored in the storage unit 35. In step S4b, if the latest determination result stored in the storage unit 35 is affirmative for the video determined to be distributed in step S3, the determination unit 32 determines that the determination item is included. , when the determination result is a negative determination, it is determined that the determination item is not included. If the determination in step S4b is negative, the process proceeds from step S4b to steps S5a and S6. If the determination in step S4b is affirmative, the process moves from step S4b to step S5b and step S7, and after step S7, the process moves to step S6.

In the case of the third pattern, for example, even if the video of actor A is being distributed, the video of actor B that is not currently being distributed is also processed by the determination unit 32 at the same time. Then, in step S3 after a predetermined time has elapsed from the start of distribution, the first distribution control unit 33 performs distribution so as to switch from the video and sound information based on the performer A to the video and sound information based on the performer B. Assume that video and audio information are determined. At this timing, that is, at the timing when the video to be distributed to the video based on actor B is decided, the determination result of affirmative or negative determination for the video based on actor B has already been obtained. Therefore, in step S4b after determining the video to be distributed in the video based on the performer B, by referring to the latest determination result stored in the storage unit 35, it is possible to determine whether or not the video contains a determination item. can determine whether In this case, after determining the video to be distributed in step S3, there is no need to perform a substantial determination process as to whether or not the video includes a determination item. Therefore, it is possible to reduce or eliminate the time difference provided between acquisition of video and audio information by the imaging device 11 and distribution thereof.

Next, the operation and effects of the above embodiment will be described.
(1) A content distribution method for live distribution of content information including video, sound information, and tactile information of an actor includes a first obtaining step, a second obtaining step, a determining step, and a distributing step. . The first acquiring step is a step of capturing video and audio information by photographing the performer. The second obtaining step is a step of obtaining the performer's tactile information. The determination step is a step of determining whether or not the image acquired in the first acquisition step includes a determination item related to switching of the tactile information to be distributed.

The distribution step is a step of distributing the acquired content information. The distributing step includes a first distributing step of determining and distributing video and audio information to be distributed, and a second distributing step of distributing tactile information corresponding to the video determined in the first distributing step. If the image determined in the first delivery step includes the judgment item, in the second delivery step, after a predetermined delay time has elapsed from the start of delivery of the video and audio information in the first delivery step, the tactile sensation Start distributing information.

According to the above configuration, when the tactile information distributed in the second distribution step is switched, a delay time is provided before the tactile information after switching is distributed. In other words, the delivery of tactile information after switching is delayed. By adjusting the tactile sensation presented to the user from the tactile sensation presentation device 14 during the delay time and controlling the operation of the tactile sensation presentation device 14 during the delay time, the impression of the tactile sensation is based on the tactile signal delivered after the delay time. can be controlled. As a result, it is possible to expand the range of expression of tactile sensations based on the tactile sensation signals distributed after the delay time. In addition, it is possible to effectively improve the user's sense of immersion and presence in the live distribution.

(2) The content delivery method comprises delivering, during the delay time, modulated tactile information obtained by modulating the tactile information delivered before and after the delay time. The content delivery method comprises switching from delivering modulated tactile information to delivering said tactile information after a delay time.

According to the above configuration, it is possible to reduce the sense of incongruity that the user experiences when the tactile sensation presented to the user by the tactile sensation presentation device 14 suddenly changes.
(3) Regarding the second pattern, there are a plurality of performers to be distributed by the content distribution method. A judgment item is that the performer appearing in the video is different from the performer from whom the tactile information currently being distributed is derived.

　When the actor from which the tactile information is derived changes, the tactile sensation presented to the user tends to change greatly, which often causes the user to feel uncomfortable. Therefore, by setting the determination items associated with the switching by the performer, it is possible to more effectively reduce the sense of incongruity that the user feels.

(4) Regarding the third pattern, the first acquisition step is a step of simultaneously acquiring a plurality of images using a plurality of imaging devices 11 . The content distribution method comprises, during the distribution step, simultaneously performing the processing of the determination step on each of the plurality of videos.

According to the above configuration, it is not necessary to perform substantial determination processing after determining the video and audio information to be distributed. Therefore, it is possible to reduce or eliminate the time difference provided between acquisition of video and audio information by the photographing device 11 and distribution thereof.

In addition, this embodiment can be changed and implemented as follows. This embodiment and the following modified examples can be implemented in combination with each other within a technically consistent range.
- The tactile information delivered in the second delivery step is not limited to the tactile signal based on the tactile information acquired from the performer. For example, the tactile information itself acquired from the performer may be distributed. Instead of the tactile information acquired from the performer, tactile information generated based on the image captured by the imaging device 11 may be distributed.

In addition, a tactile information library containing a plurality of tactile information is created in advance and stored in the storage unit 35, and tactile information selected from the tactile information library is distributed based on the image captured by the imaging device 11. You may In this case, the tactile sensation acquisition device 12 has, for example, a function of generating tactile information based on an image captured by the imaging device 11, or a function of selecting tactile information corresponding to an image from a plurality of tactile information stored in advance. using a device with The tactile sensation acquisition device 12 in this case may be part of the information processing device 15 .

· The content information is not limited to the actor's video, sound information, and tactile information. Other content information includes, for example, guidance information for superimposing a video effect that visually conveys changes in tactile information on the video displayed on the viewing device 13 .

· It may be configured so that the tactile information to be distributed can be selected based on the user's operation. For example, when a plurality of pieces of tactile information are acquired from one performer, the information processing device 15 distributes the acquired pieces of tactile information. By operating the tactile sensation presentation device 14, the user selects arbitrary tactile sensation information from a plurality of distributed tactile sensation information, and drives the tactile sensation presentation device 14 based on the selected tactile sensation information.

· In the second delivery step, if the tactile information to be delivered cannot be delivered normally, the created pseudo tactile information may be delivered. Examples of cases where the tactile information to be distributed cannot be normally distributed include, for example, a case where acquisition of the tactile information from the performer fails, and a case where the tactile information acquired from the performer contains noise. .

· The determination items used in the determination step are not limited to the items in the above embodiment as long as they are items that can determine the switching of the delivered tactile information. Moreover, a plurality of items may be determined.

· In the third delivery step, the processing of the determination step performed during the delivery step may be part of the processing in the determination step. For example, if the determination step comprises a first step and a second step that are performed in sequence, only the first step is performed during the distribution step, and the second step is performed after determining the video to be distributed. The first step is, for example, a step of determining based on a first determination item, and the second step is a step of determining based on a second determination item different from the first determination item.

・Tactile information may be edited and distributed based on the actor's state and actions, and the content of the video to be distributed. For example, when a predetermined condition is satisfied, the information processing device 15 distributes tactile information edited so that the tactile sensation presented to the user by the tactile sensation presentation device 14 is strengthened or weakened. Examples of the above-mentioned predetermined conditions include, for example, when the actor's facial expression becomes a specific expression such as a smile when the corners of the mouth are raised, when the actor's face is displayed in a close-up image, and when the actor's face is displayed in a close-up image, vibrato For example, the performer performs a specific action.

· The vibration actuator 23 is not limited to the DEA, and may be a known vibration actuator used in a tactile presentation device. Known vibration actuators include, for example, other electroactive polymer actuators (EPA: Electroactive Polymer Actuator) such as ion exchange polymer metal composite (IPMC: Ionic Polymer Metal Composite), eccentric motors, linear resonance actuators, voice coils Actuators, piezo actuators.

Claims

A content distribution method for live distribution of content information including video of an actor, sound information, and tactile information for operating a tactile presentation device,
a first acquisition step of capturing the performer and acquiring the video and the sound information;
a second obtaining step of obtaining or generating the tactile information of the performer;
a determination step of determining whether or not the video acquired in the first acquisition step includes a determination item for determining switching of the tactile information to be distributed;
a distribution step of distributing the acquired content information,
The delivery step includes:
a first distribution step of determining and distributing the video and sound information to be distributed;
a second distribution step of distributing the tactile information corresponding to the image determined in the first distribution step;
When the video determined in the first delivery step includes the determination item, in the second delivery step, a predetermined delay from the start of delivery of the video and the sound information in the first delivery step A content delivery method, comprising: starting delivery of the tactile information after a lapse of time.
During the delay time, distribute modulated tactile information obtained by modulating the tactile information distributed before the delay time, after the delay time, or before and after the delay time, or pre-created connecting tactile information for connection. and
2. The content distribution method according to claim 1, further comprising switching from distribution of the modulated tactile information or the connecting tactile information to distribution of the tactile information after the delay time.
The first obtaining step is a step of obtaining a plurality of images at the same time,
3. The content delivery method according to claim 1, wherein said content delivery method comprises simultaneously performing at least part of the processing in said determination step for each of a plurality of videos during said delivery step.
The performers are plural,
4. The content distribution method according to any one of claims 1 to 3, wherein the determination item is that the performer appearing in the video is different from the performer from which the tactile information currently being distributed is derived. .
A content distribution system used in the content distribution method according to any one of claims 1 to 4,
a photographing device that acquires the video and the sound information;
a tactile sense acquisition device that acquires or generates the tactile sense information of the performer;
a viewing device configured to be used by a user to view the delivered video and audio information;
a tactile presentation device configured to present the user with a tactile sensation based on the distributed tactile information;
an information processing device that distributes the acquired content information to the viewing device and the tactile sensation presentation device;
The information processing device is
a determination unit that determines whether or not the image captured by the imaging device includes the determination item;
a first distribution control unit that determines and distributes the video and audio information to be distributed;
a second distribution control unit that distributes the tactile information corresponding to the video determined by the first distribution control unit;
When the video determined by the first distribution control unit includes the determination item, the second distribution control unit controls the distribution of the video and the sound information from the start of distribution of the video and the sound information by the first distribution control unit. A content distribution system, wherein distribution of the tactile information is started after a predetermined delay time has passed.