CN110249631A - Display control program and display control method - Google Patents

Abstract

Display control program is broadcast live publication to the image of the real space where performing artist, obtain the three dimensional local information of the real space, user is detected for the user action to performing artist's Presenting gifts, the user action information of three dimensional local information and the user action detected based on the acquisition, the gift position that should configure the gift on the real space is calculated, and can be known with the performing artist and otherwise show the calculated gift position on the real space.

Description

Display control program and display control method

Technical field

The present invention relates to the display control programs and display control method that are broadcast live publication.

Background technique

The server of publication content is described in patent document 1 etc..

Existing technical literature

Patent document

Patent document 1: No. 5530557 bulletins of Japanese Patent No.

Summary of the invention

According to one method of the present invention, a kind of display control program is provided, comprising: display device control unit makes table The image of real space where the person of drilling is shown in a display device as the object that live streaming is issued；Acquisition unit obtains institute State the three dimensional local information of real space；Test section, detection user are used for the user action to performing artist's Presenting gifts； With gift display control section, the user that the three dimensional local information and the test section obtained based on the acquisition unit is detected is dynamic The user action information of work calculates the gift position that the gift should be configured on the real space, and with the performance Person, which can know, otherwise shows the gift position of calculating on the real space.

Other modes according to the present invention provide a kind of display control method, the reality where live streaming publication performing artist The image in space, obtains the three dimensional local information of the real space, and detection user is used for performing artist's Presenting gifts User action, and the user action information of the three dimensional local information based on the acquisition and the user action detected calculates The gift position of the gift should be configured on the real space out, and can know with the performing artist otherwise will calculation The gift position out is shown on the real space.

Detailed description of the invention

Fig. 1 is the figure for indicating the overall structure of live streaming delivery system.

Fig. 2 is the block diagram of server and user terminal.

(a) of Fig. 3 is the flow chart for indicating live streaming release processing, is (b) the aobvious of user terminal when indicating live streaming publication Show the figure of picture.

Fig. 4 is the flow chart that gift by the user and performing artist select processing.

(a) of Fig. 5 is the figure of the display picture of user terminal when indicating selection gift, is (b) to indicate selection performing artist When user terminal display picture figure, be (c) the display picture for indicating user terminal when having selected gift and performing artist Figure, be (d) figure for indicating the display picture of studio display when having selected gift and performing artist, be (e) to indicate user Make the figure of the display picture of the user terminal and studio display when the movement for launching gift.

Fig. 6 is to indicate that performing artist obtains the flow chart of the acquisition processing of the gift.

(a) of Fig. 7 is the display for indicating user terminal and studio display when performing artist picks up cat ear hair band gift The figure of picture is (b) figure for indicating the display picture of user terminal and studio display when performing artist wears hair band gift, It (c) is the figure for indicating the display picture of user terminal and studio display when performing artist faces side.

Fig. 8 is the flow chart of the processing of returning a salute from performing artist to user.

(a) of Fig. 9 is the display for indicating user terminal and studio display when performing artist holds the signature ball returned a salute The figure of picture is (b) figure for indicating the display picture of user terminal and studio display when performing artist launches signature ball, It (c) is the figure for indicating the display picture of user terminal and studio display when user receives signature ball.

(a) of Figure 10 is the display picture for indicating user terminal and studio display when being added to special efficacy to performing artist Figure, be (b) figure for indicating the display picture of user terminal and studio display when user makes the movement for launching gift, (c) it is the figure for indicating the display picture of user terminal and studio display when performing artist receives gift, is (d) to indicate carrying on the back The figure of the display picture of user terminal and studio display when showing tower on scape image.

Specific embodiment

Hereinafter, being illustrated referring to attached drawing to live streaming delivery system of the invention has been applicable in using Fig. 1~Figure 10.

(summary of live streaming delivery system)

As shown in Figure 1, live streaming delivery system 1 has: carrying out the studio 10 of performance, live streaming publication such as playing live on the spot The content-data that studio 10 obtains server 20 and watch the user terminal of the content-data issued by server 20 40,60,70.Server 20 and user terminal 40,60,70 are connected via network 2.In addition, the quantity of user terminal is without being limited thereto 2 shown in place, it is also possible to 1, can also be tens of or hundreds.

As an example, the real space in studio 10,3 performing artists A, B, C as subject are played on platform And sing melody.Certainly, the number of performing artist is not limited to 3 people, is also possible to 1 people or 2 people, can also be 4 people or more.This Outside, performing artist A, B, C is either the combination such as 1 band, the performing artist independently to work for being also possible to flock together A,B,C.Have playing device 11, loudspeaker 12, microphone 13, RGB camera 14, depth camera 15, throwing in studio 10 Shadow instrument 16 and studio display 17.

Playing device 11 reproduces the music data played, by the melody based on music data from connecting with playing device 11 Loudspeaker 12 plays out.Each performing artist A, B, C hold microphone 13, and microphone 13 includes the sound of performing artist A, B, C.As An example, RGB camera 14 are the 1st camera in live streaming delivery system 1.RGB camera 14 is to shoot function with dynamic image The digital camera head of energy.As an example, RGB camera 14 is video camera.As an example, RGB camera 14 is that display data are raw At with camera etc..RGB camera 14 is the video camera for shooting the real space that performing artist A, B, C are played.RGB camera 14 as an example, has the capturing elements such as CCD and/or CMOS, the light such as detection visible light, and export by 3 colors (red, green, blue) The display data that colour signal is constituted.As an example, RGB camera 14 shoots the subjects such as performing artist A, B, C, and exporting can Show that the ornamental of the subject taken uses data as display data in the display units such as user terminal 40,60,70.In addition, As an example, RGB camera 14 exports photographed data as display data, to be shown in studio display 17.Moreover, As an example, RGB camera 14 exports the image data being shown in large-scale display device as display data, medium-and-large-sized aobvious Showing device is set to public place, live streaming meeting-place or concert hall where user A, B, C and has large screen.In addition, RGB is imaged First 14 be not necessarily video camera, is also possible to the smart machine terminal for for example having dynamic image shooting function.In this case, It can be played and being fixed on smart machine terminal on tripod etc. and similarly be acted on video camera.

As an example, depth camera 15 is the 2nd camera in live streaming delivery system 1.As an example, depth camera 15 be infrared camera.As an example, depth camera 15 is three dimensional local information acquisition camera.Depth camera 15 Obtain itself distance i.e. depth information etc. to subject.Depth camera 15 as an example, to obtain and becoming subject The acquisition unit of distance, that is, depth information of performing artist A, B, C etc. etc..As an example, depth camera 15 is obtained respectively to as quilt Distance (depth information), the distance extremely as the performing artist B of a part of subject for taking the photograph the performing artist A of a part of body are (deep Spend information) and to as subject a part performing artist C distance (depth information).As an example, depth camera 15 It obtains to the distance, that is, depth information of each point of studio of a part etc. as subject.As an example, depth camera 15 obtain the three dimensional local information of real space as subject, including performing artist A, B and C and studio.Depth Camera 15 has the light-projecting portion of irradiation infrared ray and the infrared detection portion of detection infrared ray.Depth camera 15 is used as one Example, the depth information etc. of real space is obtained according to the time until the infrared pulse irradiated from light-projecting portion reflects Three dimensional local information.In addition, RGB camera 14 and depth camera 15 either one device, be also possible to seperated Device.

As an example, projector 16 shows gift performing artist A, B, C using gimmicks such as projection mapping technologies before the lights Present, that is, gift image.Studio display 17 is arranged in real space i.e. studio 10 and shows the display dress of image It sets, as an example, the display device in front of stage mainly can be set in a manner of visual identity by performing artist A, B, C.It drills Broadcasting room display 17 as an example is flat-panel screens, is LCD display device, organic EL display device.Also, studio is shown The image of the performance of performing artist A, B, C that the display of device 17 is taken by RGB camera 14.

Server 20 generates the live data as the content-data played by performing artist A, B, C.As an example, server 20 based on the music data from playing device 11, the voice data from microphone 13, the image number from RGB camera 14 According to etc. various data, generate for user terminal 40,60,70 issue by performing artist A, B, C progress performance live streaming number According to, and publication live data is broadcast live to user terminal 40,60,70.That is, server 20 is live to user terminal 40,60,70 The performance that performing artist A, B, C are carried out.

In addition, the performance that performing artist A, B, C are carried out can not also be reproduced by 11 pairs of accompaniments of playing device, but by The musical instruments such as the practical performance guitar of performing artist A, B, C and/or drum, and the sound is included using microphone.It can also drill It broadcasts room 10 and live data is generated by data generating device etc., and sent to server 20.

User A, B, the C for being added to live streaming delivery system 1 as an example, are the bean vermicelli of performing artist A, B, C, are able to use use Family terminal 40,60,70 watches live data.User terminal 40 as an example, have desk-top or notebook PC 40a, With the smartwatch 50 as the wearable terminal and/or smart machine terminal that are connect with PC 40a.In PC 40a be it is desk-top in the case where, user terminal 40 have desktop personal computer 40a, with the PC 40a display connecting and The smartwatch 50 being connect with PC 40a.In addition, user terminal 40 has in the case that PC 40a is notebook type The standby notebook PC 40a with display unit and the smartwatch 50 being connect with notebook PC 40a.Make As an example of, smartwatch 50 is worn on strong hand etc. by the user A of user terminal 40, smartwatch 50 and PC 40a It is connected by wired or wireless mode.Smartwatch 50 has the test sections such as acceleration transducer and/or gyrosensor, makees As an example of, when user A makes the movement for launching object, its acceleration, angle (posture) and/or angular speed are detected as user Action message.

In addition, PC 40a can also be connect by wired or wireless way with head-mounted display (HMD).In addition, HMD can have the composition as PC 40a.As HMD, optical perspective type head-mounted display, video perspective can be enumerated Type head-mounted display, non-infiltration type head-mounted display etc..It is able to carry out in the case where optical perspective type head-mounted display based on AR The display of (augmented reality).It is able to carry out in the case where video perspective type head-mounted display and/or non-infiltration type head-mounted display Display based on VR (virtual reality).In HMD, it can show and give and/or return a salute for described hereinafter The image of gift.

In addition, as an example, user terminal 60 is the smart machines terminals such as smart phone and/or tablet computer, for that can take The small information processing terminal of band.As an example, smart phone has touch panel in display surface.In addition, user terminal 60 has The test sections such as standby acceleration transducer and/or gyrosensor, as an example, when user B makes the movement for launching object, inspection Its acceleration, angle and/or angular speed are surveyed as user action information.User terminal 60 is that portable small information processing is whole End, therefore, no matter the user of user terminal 60 can wherein watch live data.

The user terminal 70 of user C has smart machine terminal 60a and smartwatch 50.In this case, smart machine is whole End 60a undertakes function as the notebook PC 40a of user A.As a result, in user C with having worn smartwatch 50 Strong hand when making the movement for launching object, be also capable of the smart machine terminal 60a's that is held in another hand of visual identity The image shown in display unit.Furthermore it is possible to smart machine terminal 60a is placed in desk etc. or is fixed on the shape of tripod Under state, the movement for launching object is carried out.Thereby, it is possible to one side to watch the shadow shown in the display unit of smart machine terminal 60a As one side is carried out launching movement etc. with the strong hand for having worn smartwatch 50.

In user terminal 40,60,70, live data can be watched on one side, and gift is virtually gifted at this time on one side Performing artist A, B, the C actually performed.As an example, it in the display surface of user terminal 40,60,70, is shown together with live data It is shown with gift selection image, gift selection image shows the work of the gift of can gift performing artist A, B, C in a manner of having a guide look of For the image of the 1st image.As gift, the ornaments such as bouquet, hair band, the demonstration when user terminal 40,60,70 is watched can be enumerated The special efficacy of the movement of performing artist, the background image in the practical place performed of performing artist etc..Also, user A, B, C are selected from gift A certain gift is selected in image guide look in image, and further selection will gift the performing artist of present from performing artist A, B, C. In fig. 1 it is shown that selecting performing artist A as performing artist, in addition, selecting cat ear hair band as the example of gift.

Then, the user A of user terminal 40 brandishes arm in the state of having worn smartwatch 50 and makes and launch object The movement of body.The user B of user terminal 60 brandishes arm in the state of holding user terminal 60 and makes and launch the dynamic of object Make.The user C of user terminal 70 brandishes arm in the state of wearing smartwatch 50 and makes the movement for launching object.This When, 40,60,70 pairs of servers 20 of user terminal are sent as testing result, that is, user action information acceleration information, angle The operation datas such as (posture) data, angular velocity data.In addition, in user terminal 60,70 live data can also be being shown Touch control operation is carried out to the direction of performing artist A, B, C of display using finger or stylus on display surface, and by the coordinate data Etc. operation datas be sent to server 20.

At this point, server 20 is using projector 16 by gift shown in the gift ID sent from user terminal 40,60,70 18 image carries out image on the ground of studio 10 and shows.The gift 18 that image is shown as an example, is shown in from user The front of performing artist shown in the performing artist ID that terminal 40,60,70 is sent.In fig. 1 it is shown that the shadow in front of performing artist A Example as showing the gift 18 of hair band.As an example, projector 16 by shown gift 18 with gift by from user A, B, the side C, i.e. performing artist A, B, C front position invest the mode in direction of performing artist A, B, C and show image.In real space In, the ground location that gift is fallen, i.e., finally the gift position of display gift 18 is the specific position determined in studio 10, It is based on the acceleration information, angle-data, angular speed as user action information sent from user terminal 40,60,70 The operation datas such as data determine.Gift position is determined by three dimensional local informations such as depth informations.Gift position is as one Example, can be by determining using the test section of depth camera 15 as three-dimensional system of coordinate of origin etc..

As an example, in the case where user A, B, C make dynamics and launch movement as gift smaller, gift is shown 18 gift position is in the front of performing artist A, B, C apart from remote position.With stronger dynamics launch is made in user A, B, C In the case where the such movement of gift, the gift position of display gift 18 is the position being closer in front of performing artist A, B, C It sets.Moreover, the gift position of display gift 18 is gift quilt in the case where user A, B, C launch gift with excessive dynamics After rebounding positioned at the wall at the rear performing artist A, B, C, the position of front or behind for performing artist A, B, C.Table as a result, The person of drilling A, B, C can visual identity go out seemingly launched from user A, B, the C for being actually not at studio 10 to the direction of oneself Gift.In the display surface of user terminal 40,60,70, performing artist A, B, C appearance for launching gift are also shown together with image Show, gift image is located at the gift image position in the image that RGB camera 14 takes.User A, B, C can also be regarded as a result, Feel identifies that the gift oneself launched reaches near performing artist A, B, C.Moreover, as an example, in user A, B, C to specific Performing artist A is made that the right or in the case that the movement of gift is launched in left side, in real space, shows the gift of gift 18 Position and user A, B, C launch the direction of gift accordingly, as the position in right side or left side for performing artist A.As An example, in the case where having launched gift to the right to performing artist A, the gift position of display gift 18 is also possible to performing artist B Or the front of performing artist C.Such gift position passes through based on the conduct testing result detected by user terminal 40,60,70 The operation datas such as acceleration information, angle-data, angular velocity data and the three dimensional local information that determines determine.

Alternatively, it is also possible to also carry out shadow same as the display surface of user terminal 40,60,70 in studio display 17 As display.

The depth camera 15 of studio 10 calculates the depth information etc. everywhere in studio 10 as an example always Three dimensional local information.Depth camera 15 as an example, extracts the personage region of performing artist A, B, C, be divided into personage region and Inhuman object area.As an example, bone site is as bone number at acquisition performing artist A, B, C respective 25 for depth camera 15 According to, and calculate the depth information of each bone site.As bone site, as an example, including right-hand man, head, neck, a left side The bone sites such as right shoulder, left and right elbow, left and right knee, left and right foot.In addition, the quantity of the bone site obtained is not limited at 25.This Outside, depth camera 15 calculates at a distance from wall and/or ground in studio 10.Herein, depth information as an example, Be from the object lens in the front of depth camera 15 or sensor cover away from measurement image position (wall of studio 10 everywhere and/or Ground is everywhere) distance.In addition, depth information as an example, is the object lens or sensor cover from the front of depth camera 15 To the distance of the bone site of the performing artist as subject.

When the right hand or left hand of performing artist A is overlapped the gift position of display gift 18, as an example, the right side of performing artist A Hand or left hand it is Chong Die with gift position based on the position of depth information when, be when performing artist A picks up gift.Then, it projects Instrument 16 shows that gift 18 not in the right hand or left hand Chong Die with gift position of performing artist A.Gift herein is hair band.In It is that performing artist A makes and will pick up hair band and be worn to movement as head.Period, server 20 is to user terminal 40,60,70 The display of display surface image the image of hair band is lifted as the right hand or left hand that are held in performing artist A, and show hair band Image is worn on the appearance on the head of performing artist A.User A, B, C can be regarded in the display surface of user terminal 40,60 as a result, Feel identifies that the hair band oneself given is seen by performing artist A, and visual identity performing artist A wears the appearance of the hair band.

After this, user A, B can worn visual identity performing artist A in the display surface of user terminal 40,60,70 The appearance played in the state of hair band.As an example, in performing artist's A aspect-oriented, make the direction of the image of hair band Also cooperate the direction of performing artist A and show.Each performing artist A, B, C's judges directed through following manner: according to from RGB The display data of camera 14 carry out the face detection of performing artist A, B, C, and calculate performing artist using depth camera 15 A, the bone site of B, C.In addition, the data of the image of display gift are also three-dimensional data.As an example, based on these data In the case where detecting that performing artist A faces side, the direction of cooperation performing artist A changes the direction of the image of hair band also.Performance The state that person A has worn hair band is also shown in studio display 17, and performing artist's A, B, C visual identity can be made to be shown in use The state of family terminal 40,60,70.

As an example, server 20 can (performing artist A, B, C be received at the time of user is to performing artist's A, B, C Presenting gifts At the time of to gift), determine all gifts and the corresponding use given to performing artist A, B, C (performing artist A, B, C are received) Family ID.Server 20 can determine the User ID of Presenting gifts at the time of performing artist A, B, C pick up gift as a result,.

When entering interlude, as an example, display is to performance in the display surface of the studio display 17 of studio 10 Person A has given the image of the image of the User ID of hair band this gift.In addition, as an example, using projector 16 by User ID The image of image be shown on the ground of studio 10.Performing artist A will appreciate that user A, B, the C for having given hair band as a result,.

As an example, when performing artist A says User ID and is indexed to microphone 13, server 20 is according to from Mike The voice data of wind 13 carries out voice recognition processing, carries out thereby, it is possible to determine the User ID of the user to return a salute to the user It returns a salute.Voice recognition processing also can (server 20 be set to studio 10 by other devices other than microphone 13 Device etc.) it carries out.In addition, as an example, as processing of returning a salute, the gift that performing artist A, B, C are worn can be touched in performing artist (such as hair band) and when making the movement for launching gift of returning a salute, the User ID for giving the gift that performing artist A, B, C are touched is determined And it returns a salute the user.Moreover, as an example, server 20 can determine that live streaming publication viewer is complete by User ID Member, and performing artist A, B, C is made to carry out processing of returning a salute to the user for the gift for having purchased certain amount of money or more.Furthermore as an example, There is user terminal to have the line of vision detecting section of detection user's sight, the time that user watches specific performing artist attentively can be calculated Situation.It in this case, can be for persistently having viewed a timing to performing artist A when performing artist A return a salute and handles Between the user of above User ID return a salute.In addition, for not to User ID more than performing artist A viewing certain time User returns a salute to the user for the User ID extracted out at random.

As above, in the case where being returned a salute, by as performing artist A to user A, B, C present returned a salute the 2nd The image of the gift image of image is shown in the display surface of studio display 17 and user terminal 40,60,70.It is specific and Speech, when the gift for the product of returning a salute is the signature ball of performing artist A, the display of display 17 and user terminal 40,60,70 in studio In face, the image of the gift image for ball of signing is shown as the right hand of performing artist A or left hand hold signature ball and is being performed The right hand of person A or the position of left hand.Performing artist A will appreciate that the status for oneself holding signature ball now as a result, in addition, user A, B, C also will appreciate that performing artist A will launch the status as the signature ball returned a salute to oneself.

When performing artist A carries out launching the movement of signature ball, depth camera 15 holds the hand of signature ball based on performing artist A The variation of depth information etc. launched signature ball to detect performing artist A.At this point, in the display surface of user terminal 40,60,70 In, gift image is shown as having launched signature ball to user A, B, C such as performing artist A.In user terminal 40,60,70 When having carried out catching the operation for the signature ball for being shown in display surface, user A, B, C receive signature ball.In addition, as returning a salute Present is not limited to signature ball.User A, B, C can when being broadcast live publication next time, by signature ball received etc. return a salute product make Performing artist A, B, C are launched for gift.Also, as the present for the product of returning a salute can also in the future by material object be mailed to user A, B, C.In mailing, actual signature ball may not be, but paper label and/or commodity relevant to performing artist and/or CD, DVD Equal albums, discount coupon of concert etc..

(depth camera 15)

Depth camera 15 as an example, has light-projecting portions such as the projector of infrared ray for issuing pulse modulated and infrared The infrared detections such as line camera portion calculates depth information according to the infrared pulse reflected time to sending (Time of Flight (TOF) mode: flight time mode).Depth camera 15 as an example, calculates studio 10 always In the three dimensional local informations such as depth information everywhere.

In addition, depth camera 15 extracts the personage region of performing artist A, B, C, it is divided into personage region and inhuman object area. As an example, same place (as an example, being studio 10), personage is calculated based on the difference value before and after shooting personage Region.In addition, as an example, the infrared ray amount that will test is more than that the regional determination of threshold value is people's object area.

Also, depth camera 15 detects bone site.Depth camera 15 obtains the depth letter in personage region everywhere Breath, the characteristic quantity based on depth and shape and calculate personage region shooting personage real space on position (left and right Hand, head, neck, right and left shoulders, left and right elbow, left and right knee, left and right foot etc.), and the center at each position is calculated as bone Position.Depth camera 15 is using the characteristic quantity dictionary being stored in storage unit, by the feature determined according to personage region The characteristic quantity for measuring and being registered in each position in this feature amount dictionary compares to calculate each position in personage region.

Alternatively, it is also possible in the following way, it may be assumed that the testing result that depth camera 15 obtains infrared detection portion is defeated Out to other devices (server 20, user terminal 40,60,70, the computing device for being set to studio 10 etc.), by other devices Calculating depth information is carried out, personage region is extracted, divides personage region and inhuman object area, detection personage region, detection bone Position determines that each position in personage region etc. is handled.

In addition, the above such motion capture processing does not carry out performing artist's A, B, C configuration flag, but can also be with To performing artist's A, B, C configuration flag.

Moreover, when calculating depth information, can also using reading the infrared line pattern issued, according to the deformation of pattern come Obtain the mode (Light Coding mode: pumped FIR laser mode) of depth information.

Further, depth information can also according to the parallax information obtained by dual-lens camera head, more cameras come It is calculated.Moreover, depth information also can be by carrying out image recognition to the image obtained by RGB camera 14, and use Photo measuring technique etc. carries out image analysis to calculate.In this case, since RGB camera 14 is used as test section to play a role, So not needing depth camera 15.

(server 20)

As shown in Fig. 2, server 20 has the interface (hereinafter simply referred to as " IF ") with each portion in studio 10, by wired Or wireless mode connection.As the IF that each portion with studio 10 is connect, has audio IF 21, RGB camera IF22, depth and take the photograph As head IF23, projector IF24 and display IF25.Moreover, having database 26, data store 27, network IF28, primary storage Device 29 and control unit 30.Server 20 issues live data to user terminal 40,60,70, is also used as control projector 16, drills The display control unit for broadcasting the display of room display 17 and/or user terminal 40,60,70 plays a role.

Audio IF 21 and the playing device 11 of studio 10, microphone 13 etc. are connect.Audio IF 21 is defeated from playing device 11 Enter to have the music data of performance, the voice data for having performing artist A, B, C is inputted from microphone 13.

The image data for the studio 10 that RGB camera IF22 input has RGB camera 14 to shoot.Depth camera IF23 Input has studio 10 everywhere and/or the depth information of performing artist A, B, C, the data in personage region, the depth of bone site letter Breath etc..Projector IF24 carries out showing the control of gift on ground of the stage of studio 10 etc. by projector 16.Show IF25 Control is set to the studio display 17 of studio 10.As an example, display surface of the display IF25 in studio display 17 It is middle to show the gift image given to performing artist A, B, C and/or User ID.It is who gives that performing artist A, B, C, which is able to know that, as a result, Gift.

Database 26 as management department is built by every live streaming with the User ID with the user registered in this system Associated mode is found to manage the gift of each user.Specifically, in database 26, be broadcast live by every, with each User ID Establish associated mode manage gift ID, gift send out destination ID, gift whether received, whether there is or not return a salute and whether at Function, which receives, returns a salute.

Gift ID is the ID for uniquely determining the gift that user is bought, and is to uniquely determine user institute in each live streaming The ID for the gift given.

It is the ID for uniquely determining the performing artist for being given away gift that gift, which sends out destination ID,.

Whether gift is received whether the performing artist that management user selects successfully has collected the gift that user gives out.

Whether the performing artist for having collected the gift that user gives whether there is or not management of returning a salute has carried out time the user for sending out present Gift.

Whether have successfully received to return a salute to manage whether successfully to have collected and return a salute.

In addition to this, database 26 will participate in all users of live streaming publication and User ID foundation associatedly carries out pipe Reason.The user for participating in each live streaming selects from all users of registration.In addition, database 26 is as an example, by the valence of each gift Lattice are associatedly managed with gift ID foundation.In addition, as an example, managing the purchase amount of money corresponding with performing artist of each user Total value.

User A is shown in FIG. 2 and has given gift ID " A (bouquet) " to performing artist C, but present is not by performing artist C The situation collected.And show user B and given gift ID " C (hair band) " to performing artist A, performing artist A has collected gift A, and And performing artist A is returned a salute, and user A has collected the situation returned a salute.It also shows user C and has given gift to performing artist B Product ID " B (special efficacy) ", performing artist B has collected gift B, and performing artist B is returned a salute, but user C could not successfully collect this The situation returned a salute.

Data store 27 is the storage devices such as hard disk.Data store 27 is preserved relevant to live streaming delivery system 1 Control program and/or the display data of the image for showing gift etc..For showing the display data conduct of the image of gift An example is corporeal thing in gift, such as is three-dimensional data when ornament, is configured to cooperate the direction of performing artist and show decoration Product.As an example, the data of ornament are shown when performing artist faces front from the front of ornament, in performing artist face When towards side, ornament is also displayed as towards side.It controls program as an example, is to issue to be broadcast live to user terminal 40,60,70 The release process of data.It controls program as an example, is to be shown the image for the gift given from user using projector 16 Gift display control program on the ground of studio 10.In addition, control program is as an example, it is the gift that will be given from user Image and performing artist A, B, C, which are established, associatedly carries out the display surface of studio display 17 and/or user terminal 40,60,70 The display device of display controls program.In addition, control program is as an example, it is whole for studio display 17 and/or user The display surface at end 40,60,70 shows that the display device of the image of the User ID of benefactor controls program.Moreover, control program is made As an example of, it is when the performing artist for being given away gift returns a salute user gift, to studio display 17 and/or user The display surface of terminal 40,60,70 shows the display device control program for the gift image returned a salute.

Network IF28 is connect via the networks such as internet 2 with server 20 and user terminal 40,60,70.Main memory 29 It as an example, is RAM, live data and/or control program in temporarily storage publication etc..

Control unit 30 is CPU, the whole movement of control server 20 as an example.Control unit 30 as an example, be according to Publication control program issues the distribution unit of live data to user terminal 40,60,70.Control unit 30 is according to gift as an example The gift sent out from user is shown that the gift on the ground of studio 10 is shown using projector 16 by product display control program Control unit.Control unit 30 is the display device control unit for controlling the display of user terminal 40,60,70, in addition, being as an example Control the display device control unit of the display of studio display.

Such control unit 30 is generated and is shown to the display surface of studio display 17 and/or user terminal 40,60,70 The gift image sent out from user and performing artist A, B, C are established into the display data associatedly shown.Moreover, as an example, it is right The display surface of studio display 17 and/or user terminal 40,60,70 generates and shows the display of the User ID of display benefactor Data.Furthermore control unit 30 is as an example, when the performing artist for being given away gift returns a salute user gift, to drilling The display surface for broadcasting room display 17 and/or user terminal 40,60,70 generates and shows the display data for the gift returned a salute.

When gift image to be shown to the display surface in studio display 17 and/or user terminal 40,60,70, show Show the gift image position that should have in the real space of studio 10.That is, practical sky of the gift position in studio 10 Between in position determined by three dimensional local information.About in studio display 17 and/or user terminal 40,60,70 The image of the gift shown on display surface is also shown in and obtains under the direction even if the direction of RGB camera 14 changes Gift image appropriate position in the image taken.In addition, the case where gift position exceeds the coverage of RGB camera 14 Under, gift image is no longer shown in the display surface of studio display 17 and/or user terminal 40,60,70.Also, it is drilling In the display surface for broadcasting room display 17 and/or user terminal 40,60,70, gift image squats down or takeoffs in performing artist A, B, C In the case of, it shows with also cooperating the movement of performing artist A, B, C.

Control unit 30 does not need to carry out such all processing above, can also cooperate with other devices and carry out a part Processing.As an example, can be configured to that PC equal controller is arranged in studio 10, by the control for being set to studio 10 The collaboration of device and server 20 processed carries out such system handled above.In this case, for example, server 20 has database 26, main memory 29 and control unit 30.In addition, control device has audio IF 21, RGB camera IF22, depth camera IF23, projector IF24, display IF25, data store 27 and network IF28.Also, control device as an example, can be made Carry out the processing in addition to the update of database 26.For example, showing the image for the gift given by user using projector 16 Processing, and/or shown in the display surface of studio display 17 and/or user terminal 40,60,70 gift image processing Deng.

Such a part handled above is carried out in addition it is also possible to cooperate with user terminal 40,60,70.For example, will be by Image data, the three dimensional local information obtained by depth camera 15 of real space etc. that RGB camera 14 obtains are sent to User terminal 40,60,70.User terminal 40,60,70 detects the movement of user A, B, C, image data based on real space, The testing result of the movement of three dimensional local information, user A, B, C, and show gift to eventually arriving at position in the display surface of itself It is set to track, the display of gift image etc. only.

(user terminal 40)

User terminal 40 as an example, be user A management device, have desk-top or notebook PC 40a and Smartwatch 50.As an example, notebook PC 40a has audio IF 41, display IF42, network IF43, communication IF44, data store 45, operation IF46, main memory 47 and control unit 48.Audio IF 41 and loudspeaker, earphone, wear-type The connection of the audio input devices such as the audio output devices such as earphone and/or microphone.IF42 is shown as an example, and by liquid crystal display The display unit 49 that the display devices such as device are constituted connects.

Network IF43 is communicated with server 20 as an example, via network 2.Communication IF 44 as an example, with smartwatch 50 communications.Communication IF 44 is connected with smartwatch 50 by Wireless LAN and/or wired lan, has conduct from the input of smartwatch 50 Acceleration information, angle-data, angular velocity data of user action information etc..Data store 45 is nonvolatile memory, It as an example, is hard disk and/or flash memory.Data store 45 is stored with the reproduction program and/or and smartwatch of live data 50 communication control program etc..Operation IF46 is connect with operating devices such as keyboard and/or mouses.In addition, connecting with display IF42 When being provided with touch panel in the display surface of the display unit 49 connect, operation IF46 is connect with the touch panel.Main memory 47 is made As an example of, it is RAM, live data and/or control program in temporarily storage publication etc..Control unit 48 is CPU as an example, Control the whole movement of user terminal 40.As an example, control unit 48 is when reproducing live data, among performing artist A, B, C 1 people or more people are selected, performing artist is sent to server 20 and selects data, in addition, being sent in the guide look of gift image to server 20 One or more gifts select data.Moreover, control unit 48 is as an example, server 20 is sent and is detected by smartwatch 50 The operation datas such as the acceleration information, angle-data, angular velocity data as user action information arrived.

Smartwatch 50 is the information processing of Wristwatch-type equal in the wrist for be worn on the strong hand of user A as an example Terminal.Smartwatch 50 has sensor 51, communication IF 52, data store 53, main memory 54 and control unit 55.Sensing Device 51 is such as acceleration transducer and/or gyrosensor.Communication IF 52 as an example, is sent to PC 40a by passing The angle-data and/or angular velocity data of acceleration information, smartwatch 50 that sensor 51 detects.Sensor 51 is used as one Example, when user A makes the movement for launching object, from communication IF 52 to PC 40a, transmission is brandished relevant to its arm The operation datas such as acceleration information, angle-data, angular velocity data as user action information.Data store 53 is non-easy The property lost memory is as an example hard disk and/or flash memory.Data store 53 be stored with driving sensor 51 driver and/ Or with the communication control program of PC 40a etc..Control unit 55 is CPU as an example, and control smartwatch 50 is whole to be moved Make.

In addition, the terminal connecting with user terminal 40 may not be smartwatch 50, but has acceleration transducer And/or smart phone of gyrosensor etc. is small-sized and the portable information processing terminal.

(user terminal 60)

User terminal 60 is that smart phone and/or tablet computer etc. are intelligently set for the device of user B management as an example Standby terminal.User terminal 60 as an example, has audio IF 61, display IF62, operation IF63, sensor 64, network IF65, number According to storage unit 66, main memory 67 and control unit 68.The audio output devices such as audio IF 61 and boombox and/or earphone And/or the audio input devices such as built-in microphone connection.Audio IF 61 is broadcast from audio output device as an example, by live data It puts.Display unit 69 display IF62 small-sized with built-in liquid crystal display panel, organic EL panel etc. is connect.Display unit 69 is provided with touching Panel is controlled, operation IF63 is connect with touch panel.Sensor 64 is such as acceleration transducer and/or gyrosensor.Network IF65 is communicated with server 20 as an example, via network 2.Network IF65 as an example, makes in user and launches the dynamic of object When making, send that sensor 64 detects to server 20 brandishes relevant acceleration as user action information and arm The operation data of data, angle-data, angular velocity data.In addition, showing live data using finger and/or stylus Display surface on when carrying out to the slide of the direction touch control of performing artist A, B, C of display, network IF65 is sent out to server 20 Give the operation datas such as its coordinate data.Data store 66 is nonvolatile memory, is flash memory as an example.Data storage Portion 66 is stored with reproduction program of live data etc..Main memory 67 is RAM as an example, the live streaming in temporarily storage publication Data and/or control program etc..

Control unit 68 is CPU, the whole movement of control user terminal 60 as an example.As an example, control unit 68 is again When existing live data, 1 people or more people are selected among performing artist A, B, C, and selection data are sent to server 20, in addition, right Server 20 sends one or more the selection data in the guide look of gift image.Moreover, control unit 48 is as an example, in user When making the movement for launching object, the acceleration information brandished, the angle of the arm are sent to server 20 for handset user terminal 60 The operation datas such as degree evidence, angular velocity data, coordinate data.

Hereinafter, being illustrated to the effect of live streaming delivery system 1.

(live streaming release processing)

Before live streaming publication, firstly, making depth camera 15 obtain the depth everywhere in studio 10 in studio 10 Information is spent, and calculates personage region, then, calculates bone site in personage region, and each bone position can be calculated The depth information set.Then, motion capture processing is executed in depth camera 15.In addition, making user terminal 40,60,70 one-tenth For that can log in server 20, and watch the state of live streaming publication.

As shown in (a) of Fig. 3, when performing artist A, B, C strike up, in step sl, server 20 is generated in conduct Hold the live data of data.It is being drilled specifically, inputting performing artist A, B, the C for thering is RGB camera 14 to take in server 20 Broadcast the image data for the real space played in room 10.In addition, having melody number from the input of playing device 11 in server 20 According to inputting from microphone 13 has the voice data of performing artist A, B, C.Then, server 20 is based on these various data, generates and uses In the live data for the performance that performing artist A, B, C for issuing to user terminal 40,60,70 are carried out.In addition, in step s 2, The input of server 20 has studio 10 everywhere and/or the depth information of the bone site of performing artist A, B, C.In step s3, it takes Device 20 be engaged in the live streaming publication live data of user terminal 40,60,70.That is, server 20 in real time fashion to user terminal 40, 60, the performance that 70 publication performing artist A, B, C are carried out.As a result, as shown in (b) of Fig. 3, user terminal 40,60,70 is in display surface In show the live views 71 based on live data, in addition, output live sound.

(gift/performing artist's selection processing)

Next, being illustrated using Fig. 4 to gift/performing artist's selection processing.Herein, user terminal is passed through with user B 60 watch live data and are illustrated in case where being operated.In step s 11, server 20 is in user terminal 60 Gift selection image 72 (referring to (a) of Fig. 5), the gift candidate are shown in display surface in a manner of Chong Die with live views 71 The image for showing in a manner of having a guide look of selectable gift as 72.As an example, in (a) of Fig. 5, gift is selected in image 72, There are in a manner of successively forming a line from a left side indicates that the image 72a of bouquet gift, addition are demonstrated in the movement of performing artist Special efficacy gift image 72b, indicate cat ear hair band gift image 72c and indicate the background image of live streaming publication The image 72d of gift.

The gift enumerated in gift selection image 72 is the multiple gifts prepared by one side of network operator.The gift of preparation was both Can be different according to every live streaming, it is also possible to general in all live streamings.In addition it is also possible to be the gift of a part more It is repeated in a live streaming.In the database of server 20, the price of each gift and gift ID foundation are associatedly managed.Clothes Device 20 be engaged in as an example, is established with gift ID and associatedly saves dynamic image data, image data, voice data and/or melody Data etc. are as showing the gift data of the gift.Gift data are three-dimensional data as an example.

Each gift is charge, and the amount of money is determined according to gift, and price is associated with gift ID foundation.As an example, gift ID is that the gift of the bouquet of " A " is 200 yen.Addition gift ID is that the gift of the special efficacy of " B " is 300 yen.Gift ID is " C " Cat ear hair band gift be 500 yen.Gift ID is that the gift " D " of the background image of " D " is 1000 yen.As an example, User A shown in the database of Fig. 2 has purchased the bouquet of gift ID " A " with 200 yen, and user B has purchased gift with 500 yen The hair band of product ID " C ", user C have purchased the special efficacy of gift ID " B " with 300 yen.User A, B, C can be like this via users Terminal 40,60,70 buys these gifts, thus to performing artist's A, B, C Presenting gifts.Performing artist A, B, C and network operator's energy as a result, Access sales volume corresponding with the gift that user A, B, C give.No matter whether performing artist A, B, C collect (such as picking up) gift Product, all gifts that user A, B, C are given become the sales volume of performing artist A, B, C and its network operator.It, can also in gift To there is free gift.In addition, 1 user can both buy 1 gift, can also buy multiple gifts in 1 live streaming. In the database, the total value of the purchase amount of money corresponding with performing artist of each user is managed.As a result, as an example, make network operator institute The server 20 of management is able to carry out the processing etc. for making performing artist preferentially return a salute the user for having purchased more gift.

When user B has selected 1 gift from the guide look that gift selects image 72 by user terminal 60, user terminal 60 pairs of servers 20 send the gift including the gift ID of User ID and selected gift and select data.In step S12 In, server 20 carries out the selection processing of gift based on gift selection data.Server 20 sends user terminal 60 only will choosing The image 72c for the gift selected is shown in the selection data of user terminal 60, in the display surface of user terminal 60 with live shadow As the mode of 71 overlappings shows image 72c.In (b) of Fig. 5, as an example, showing the selection of server 20 indicates hair band The image 72c of gift, and it is shown in the state in the downside corner of the display surface of user terminal 60.Correspondingly, server 20 In order to also be now arranged in the state of gift selection processing to performing artist A, B, C notice, in studio, display 17 is also carried out equally Display.

In addition, server 20 is in the display surface of user terminal 60, as an example, display by performing artist A, B, C everyone The performing artist respectively included selects image 73.Herein, in the display surface of user terminal 60, as an example, accordingly display is logical The 1st prompt image 73a of performing artist's selection operation will be carried out by knowing user B next.1 performance is being selected by user terminal 60 When person selects image 73, user terminal 60 sends including the performing artist ID of User ID and the performing artist of selection server 20 Performing artist select data.In step s 13, the selection that server 20 selects data to carry out performing artist based on performing artist is handled. Server 20 shows in the display surface of user terminal 60 the performing artist A for selection in a manner of Chong Die with live views 71 Performing artist determines image 74.In (c) of Fig. 5, as an example, shows server 20 and selected performing artist A, and be shown in use State in the display surface of family terminal 60.Performing artist selects image 73 and/or performing artist to determine that image 74 can be quadrilateral shape, However, not limited to this, being for example also possible to circular shape or triangle.Correspondingly, as shown in (d) of Fig. 5, server 20 in order to also to performing artist A, B, C notice be now arranged in performing artist select processing state, on studio display 17 also into The same display of row.In studio display 17, display indicates that user B wants to send out the 2nd prompt figure of selected gift As 74a.

In addition, server 20 registers selected gift and table in database 26 after having selected gift and performing artist The person of drilling.

Then, the user of user terminal 60, which becomes, can operate user terminal 60 and to the performing artist for being in studio 10 A, the state of B, C Presenting gifts.Specifically, the user B of user terminal 60 can pass through handset user terminal 60 and make throwing The movement of object out, and obtain the pseudo-experience that the performing artist for selecting the gift of selection to oneself launches.Specifically, service Device 20 and user terminal 60 start synchronization process, and user terminal 60 sends by sensor 64 server 20 by per unit time The operation datas such as the acceleration information, angle-data, angular velocity data as user action information detected.In step S14 In, server 20, which is stored with, to be judged as judging whether user has carried out the threshold value for launching movement when being more than threshold value User terminal 60 has carried out launching movement.As an example, server 20 in order to determination launch movement, be stored with acceleration information, The threshold values such as angle-data, angular velocity data.Server 20 has been more than threshold in acceleration information, angle-data, angular velocity data etc. When value, it is judged as and has carried out launching movement.In addition, the distance etc. of the beginning and end when carrying out the slide of touch panel When being more than threshold value, it is judged as and has carried out launching movement.In addition, in the case where user terminal 40, in acceleration information, angle When data, angular velocity data etc. have been more than threshold value, it is judged as and has carried out launching movement.

In step S15, server 20 brandishes relevant acceleration to arm based on sending from user terminal 60 The operation datas such as data, angle-data, angular velocity data, parse the arm of user brandishes direction and/or speed etc..As a result, Server 20 calculates track and/or lowering position i.e. gift position when the gift launched is launched.As an example, gift position Setting can be determined by three-dimensional system of coordinate using the test section of depth camera 15 as origin etc..

In step s 16, server 20 based on analysis result, generates what expression was shown in the display surface of user terminal 60 The display data for falling image 75 of the gift of hair band are fallen, and user terminal 60 is sent.As a result, as shown in (e) of Fig. 5, In the display surface of user terminal 60, display in real time as fallen image 75 to performing artist A.In addition, to studio 10 studio display 17 sends the display data for similarly falling image 75, also shows in real time in studio display 17 Show and falls image 75.Image 75 is fallen when being shown in the display surface of studio display 17 and/or user terminal 60, is shown in The gift image position that should have in the real space of studio 10.That is, the position about gift, the practical sky of studio 10 Between in gift position determined according to three dimensional local information.Thus, even if the direction of RGB camera 14 changes, fall Image 75 is also shown in the suitable gift image position in the image obtained under the direction of the RGB camera 14.In addition, In the case that gift position has exceeded the coverage of RGB camera 14, falls image 75 and no longer show.

In addition, the processing that display falls image 75 in studio display 17 can be by following steps S17, benefit The display of image 75 will be fallen on the ground of studio 10 with projector 16, therefore can be omitted.

In step S17, server 20 sends the display data for falling image 75 to projector 16, and projector 16 is being performed in a radio or TV programme On the ground of room 10, real-time display the image of gift that comes and/or drops down onto gift position to performing artist A with the state circled in the air The image of gift.Performing artist A, B, C will appreciate that the lowering position for falling image 75 as a result,.

In addition, as long as falling image 75 is displayed at least gift position, until the midway circled in the air until gift position State and/or track can not also show.

In addition it is also possible to which it is aobvious to be not at the mode in the movable performing artist's scope of activities of performing artist A, B, C with gift position Show and falls image 75.Furthermore, it is possible to enter image in performing artist's scope of activities during until reaching gift position, but It can be so that the mode that the image for the gift that user gives does not enter in performing artist's scope of activities finally shows and falls image 75. In addition, even if detecting user to the movement (such as launching) of performing artist's Presenting gifts and calculated gift position is located at performance In person's scope of activities, image can not also be shown in performing artist's scope of activities.In this case, for example, it is contemplated that calculated Gift position, can outside performing artist's scope of activities near (outside performing artist's scope of activities, and away from calculated gift position Nearest position) display image.According to such display mode, it can prevent performing artist A, B, C from accidentally stepping on the gift given.

In addition, performing artist's scope of activities is as an example, for the stage etc. of studio 10 etc..In addition, performing artist's scope of activities Can also in 1 song prelude, interlude and during caudal flexure and in addition to this during set different ranges.In addition, can also (such as it is set as performing artist during playing music with happy to set different ranges during playing music and during non-performance Bent and movable range, is not set as not having performing artist's scope of activities during playing music).Moreover, both can be according in performance Melody and set different ranges, can also be broadcast live publication during be set as same range always.It is not limited to play, Different ranges is set during can be during performance and before and after performance.

Moreover, can both gift be made to fall at this if gift meets performing artist, or change the heading of gift Become, can also move it.

(gift acquisition processing)

Next, being illustrated referring to Fig. 6 to the performing artist A movement for obtaining the hair band gift that user B gives.

In server 20, the bone site for having each performing artist A, B, C and each bone position are inputted from depth camera 15 always The depth information set.In addition, server 20 carries out the face detection of performing artist A, B, C according to the image from RGB camera 14. Server 20 tracks each bone site, the depth information of the bone site and the facial position of performing artist A of performing artist A as a result, It sets.In the step s 21, server 20 judges that performing artist A has picked up the gift of hair band.As an example, server 20 is for determination The movement for dropping down onto the gift on ground of studio 10 is picked up, the bone site of left and right hand and the bone of left and right foot are stored with The distance of bone position and/or the bone site of left and right the hand relevant threshold value such as at a distance from ground.Server 20 is calculating The bone site of left and right hand out is at a distance from the bone site of left and right foot and/or the bone site of left and right hand When being more than threshold value with the data such as at a distance from the ground of gift position, it is judged as that performing artist A has picked up the gift of hair band.As one Example, server 20 determine that performing artist A has picked up hair band gift at the position of left and right hand and the Chong Die gift position on ground Product.In other words, judge whether performing artist is located in the range of the image for the gift given from user to performing artist.The image of gift Range as an example, determined according to the type of three dimensional local information, user action information, gift.As an example, in hair band In the case where, it is the range of the outer shape of hair band.The position of left and right hand with whether the gift position on ground be overlapped sentences Break as an example, whether the three-dimensional information (1 point) for judging the position of the hand of performing artist and the three-dimensional information (1 point) of gift position weigh It is folded.In addition, as an example, judging the three-dimensional information (multiple spot) of the position of the hand of performing artist and the three-dimensional information (1 of gift position Point) whether it is overlapped.Further, as an example, judge the three-dimensional information () and the three of gift position of the position of the hand of performing artist at 1 point Whether dimension information (multiple spot) is overlapped.Further, as an example, judge three-dimensional information (multiple spot) and the gift of the position of the hand of performing artist Whether the three-dimensional information (multiple spot) that grade is set is overlapped.Further, as an example, judge the three-dimensional information of the position of the hand of performing artist Whether (regions such as finger tip) and the three-dimensional information (region that display falls image 75) of gift position are be overlapped.In the following way more It is easy to carry out the judgement whether be overlapped with the gift position on ground of the position of left and right hand, it may be assumed that be not the hand for judging performing artist Position three-dimensional information and the three-dimensional information of gift position both whether be overlapped, but judge at 1 point whether have multiple spot and/or Region overlapping.

In step S22, server 20 carries out making to be shown in falling what image 75 was not shown on the ground of studio 10 Control.Because if gift is picked up by performing artist A, then gift can disappear from ground.

During being picked up from gift by performing artist A to the head for being worn on performing artist A, by performing artist A it is hand held in.Make As an example of, (a) of Fig. 7 shows the state that performing artist A picks up the gift of hair band.Then, in step S23, server 20 is solved Analyse the acquisition movement of performing artist A.That is, server 20 according to each bone site of performing artist A, the bone site depth information, With the position of the face from performing artist A, the movement until the gift of hair band is worn on head is parsed.In step s 24, it services Based on analysis result, generate in the display surface of studio display 17 and user terminal 60 indicates from picking up to being worn on device 20 The display data of the acquisition image 76 of hair band gift until head, and studio display 17 and user terminal 60 are sent. As a result, as an example, in the display surface of studio display 17 and user terminal 60, image 76 is obtained in the gift from ground During position is moved to until head, is established with pickup with hand and associatedly shown.

In step s 25, server 20 parses performing artist A and acts the wearing that the gift of hair band is worn on head.That is, clothes Device 20 be engaged according to the facial positions of each bone site of performing artist A, the depth information of the bone site and performing artist A, parsing will The gift of hair band is worn on the wearing movement on head.As an example, server 20 is in the position of left and right hand and the position on head When setting overlapping, detect that wearing acts.In step S26, server 20 is in the aobvious of studio display 17 and user terminal 60 Show in face, generates the display data that the wearing position display on the head of performing artist A, B, C obtains image 76, and aobvious to studio Show that device 17 and user terminal 60 are sent.As an example, server 20 generates in a manner of along the boundary part of color development and background Display obtains the display data of image 76.As a result, as an example, in the display surface of studio display 17 and user terminal 60 Show the state for obtaining the head that image 76 is worn on performing artist A (referring to (b) of Fig. 7).Server 20 tracks performing artist A Head, and shown in a manner of always adorning oneself with the gift of hair band performing artist A movement.

Performing artist A faces side according to choreography sometimes.Even if in this case, server 20 also cooperates performing artist The direction ground display of A obtains image 76 (referring to (c) of Fig. 7).The direction of each performing artist A can be by imaging according to from RGB First 14 display data carry out the face detection of performing artist A, and come from the bone site that depth camera 15 calculates performing artist A Judgement, shows that the data of the image of gift are also three-dimensional data, also can be realized the display from any direction.Based on these numbers According in the case where detecting performing artist A as side is faced, the direction of cooperation performing artist A makes the direction of the image of hair band Variation.Also, image 76 is obtained in the case where performing artist A squats down or takeoffs, is shown with also cooperating the movement of performing artist A.

In addition, server 20 is registered in database 26 when the performing artist by selecting obtains gift, by successful intention In.

When the gift of hair band is worn on performing artist A, in step s 27, server 20 in studio display 17 and The ID image for indicating to have given performing artist A the User ID of the user B of gift of hair band is shown in the display surface of user terminal 60 76a.As a result, performing artist A can visual identity given hair band gift user User ID, in addition, user B can also lead to Cross the shown User ID of oneself and hair band gift that visual identity oneself is given is worn on performing artist A.In addition, server 20, which also can be used projector 16, is also shown in ID image 76a on the ground of studio 10.

The image of gift and performing artist are established during associatedly showing, either being broadcast live hair from obtaining The entire period of cloth can also be shown by every 1 song.In addition it is also possible to not shown during interlude.

In addition, performing artist, which takes, has worn primary gift, such as box can be stored or is placed in (either setting In the object of the physical presence of studio etc., it is also possible to the same virtual image with gift) in.Giving performing artist as a result, In the case where having given many gifts, performing artist can wear multiple presents.Such as performing artist can once wear multiple hair bands, The hair band that performing artist wears before can also be removed and pick up new hair band to wear.In this case, as desk and/or receipts Case of receiving uses box like that, demonstrates the replacement operation etc. of hair band with capable of having no abnormal sense.

(processing of returning a salute)

During interlude, performing artist A, B, C without sing, therefore, the user B for having given gift can be carried out as return The movement of report returned a salute.In step S31, server 20 judges whether the melody in live streaming performance enters interlude.Server 20 as an example, can during the prescribed period voice input not from microphone 13 when, be judged as and enter interlude.In addition, Server 20 as an example, can from playing device 11 input have the detection signal for having indicated entry into interlude when, be judged as into Interlude is entered.In addition, as an example, the movement of interlude can have been indicated entry by detection to determine to enter interlude.Service Device 20 as an example, with user terminal 60 start synchronization process, the display being able to detect in user terminal 60 and/or by user end The receiving processing of returning a salute of 60 operation of end.

In addition, processing of returning a salute can not also carry out during interlude, and carried out between song and song.In addition, waving Whens platform etc. carries out drama or musical play etc., be also possible between N acts and N+1 acts etc..

It, can be defeated when having from the voice input of microphone 13 and/or from playing device 11 in addition, terminate about interlude When having entered to indicate the detection signal that interlude terminates, determine that interlude terminates.Furthermore it is possible to indicate that interlude terminates dynamic by detection Make, determines that interlude terminates.

In above-mentioned steps S27, server 20 shown in the display surface of studio display 17 and user terminal 60 to Performing artist A has given the User ID of the user B of the gift of hair band.Thus, performing artist A can be called out out to microphone 13 to performance Person A has given the User ID of the user B of hair band gift.At this point, in step s 32, server 20 is indexed to by microphone 13 Voice data carry out voice recognition, determine the User ID of user B.In addition, server 20 is for becoming the user of object of returning a salute, The intention returned a salute is registered in database 26.

In step S33, server 20 detects the movement of returning a salute of performing artist A.As an example, detection indicates table to server 20 The person of drilling will transit to the characteristic specific action for movement of returning a salute.As an example, the storage of server 20 is for judging specific action Each bone site threshold value, each bone site data be more than threshold value when, be judged as that performing artist A has carried out specific action. Herein, product are returned a salute as an example to what user B was carried out from performing artist A, is the signature ball of performing artist A, performing artist A is in specific action Later, the movement for launching signature ball to the user B for being actually not at studio 10 from studio 10 is made.In step S34, Server 20 is parsed according to the facial positions of each bone site of performing artist A, the depth information of the bone site, performing artist A It returns a salute movement.In step s 35, server 20 as an example, in the display surface of studio display 17 and user terminal 60 It generates in the beginning for launching movement that performing artist A makes, shows signature ball in the position of the left and right hand of performing artist A It returns a salute the display data of image 77, and studio display 17 and user terminal 60 is sent.As a result, as shown in (a) of Fig. 9, As an example, real-time display is returned a salute image 77 in the display surface of studio display 17 and user terminal 60.In addition, service What device 20 generated the hand that analog subscriber B is shown in the display surface of studio display 17 and user terminal 60 receives image 78 It shows data, and studio display 17 and user terminal 60 is sent.It is virtual when launching signature ball for receiving image 78 Target.

In step S36, what the parsing of server 20 performing artist A was carried out launches movement.Specifically, server 20 is according to table Each bone site of the person of drilling A, the depth information of the bone site, performing artist A facial positions detect the arm of performing artist A It brandishes.In step S37, server 20 in the display surface of studio display 17 and user terminal 60, launching by generation The position for acting the left and right hand of the performing artist A of midway shows the display data for image 77 of returning a salute.In addition, generating display from a left side Right certain hand leaves and is in the display data of the image 77 of returning a salute for the state of circling in the air.Then, to studio display 17 and user Terminal 60 is sent.As a result, as shown in (b) of Fig. 9, as an example, the display surface of display 17 and user terminal 60 in studio In, it displays in real time to seem the appearance for having launched ball of signing to the direction for receiving image 78.

Arm alternatively, it is also possible to launch signature ball to performing artist A adds special efficacy.As an example, which is detection table The movement for the person of drilling and special efficacy corresponding with the movement detected are in the moving direction downstream of the arm of mobile performing artist A The edge of side shows the special efficacy of multiple twinkLing stars type figures.As an example, special efficacy detect return a salute act at the time of, with table The arm foundation for launching signature ball of the person of drilling A is associatedly shown.Then, be changed into launch movement when, cooperate and brandish Moving for arm and the edge for being shown in the moving direction downstream side of arm.In addition it is also possible to detect movement of returning a salute when It carves, background image is changed into the specific image shown when returning a salute and handling.

When receive operation at the time of the arrival of image 77 of returning a salute receives image 78 using user terminal 60, user is whole 60 pairs of transmissions of server 20 are held to receive data including User ID.Receiving operation is to embrace the movement of ball, is being drawn with mouse Any position in face and/or receive the operation clicked at image 78.Receive operation as an example, is to touch touch panel Operation.In step S38, server 20 receive receive data when, in database 26 registration have received the intention returned a salute.This When, server 20 generates display and is caught by receiving image 78 in the display surface of studio display 17 and user terminal 60 The display data of the state for image 77 of returning a salute, and studio display 17 and user terminal 60 are sent.As a result, such as Fig. 9 (c) shown in, as an example, in the display surface of studio display 17 and user terminal 60, show that useful hand has caught signature The state of ball.

In addition, sending out when not carrying out receiving operation at the time of the arrival of image 77 of returning a salute receives image 78 server 20 Send the miss data that receives including User ID, server 20 receive receive miss data when, registered in database 26 The intention for the receiving failure returned a salute.

(other gifts)

The image 72b for the gift of special efficacy demonstrated in the movement of performing artist shown in (a) as addition Fig. 5 is right The following such special efficacy of performing artist A, B, C addition.In (a) of Figure 10, special efficacy image 81 is added to the performing artist A of selection.It closes In the image of the special efficacy image 81, the special efficacy corresponding with the movement detected as the movement for detecting performing artist, mobile The edge in the moving direction downstream side of the arm of performing artist A shows multiple twinkLing stars type figures.Such special efficacy image 81 It is not corporeal thing as hair band described above.As shown in (b) of Figure 10, when launching to the performing artist A of selection, gift is used The gift box image 82 equipped with silk ribbon etc. of object.Also, as shown in (c) of Figure 10, when the performing artist A by selecting is obtained, i.e., It when the position of left and right hand is Chong Die with gift position, shows that gift box image 82 no longer, later, carries out special display effect image 81 Control.

As an example, it even if in the case where performing artist A squats down or takeoffs, shows with also cooperating the movement of performing artist A.Make As an example of, special efficacy can be made to change before and after performing artist A, B, C jump.As an example, Special display effect flashing before jumping Star-like figure glisteningly shows different figures after jump.In addition, as an example, pre-defining multiple specific actions, detecting When to 1 specific action, it is added the display of specific special efficacy associated with the movement.In addition, as an example, detecting When to specific action, stop the display of addition special efficacy.In addition, as an example, until detecting specific action, without addition The display of special efficacy.

(d) of Figure 10 shows the state for having selected to indicate the image 72d of the gift of background image of live streaming publication.At this In the case where the image 72d of the background image of sample, nor the corporeal thing as hair band above.Therefore, in the table to selection , it is preferable to use the gift box image 82 of present when the person of drilling A is launched.

Alternatively, it is also possible to also show present when launching to the performing artist A of selection in the gift for having selected hair band Gift box image 82.

In addition, image 77 of returning a salute can only show the user terminal 40,60,70 for the user to be returned a salute.Thereby, it is possible to realities Existing performing artist exchanges with 1 Dui 1 of user.

(other gifts/performing artist's selection processing)

In the example above, the case where having selected performing artist A to user B is illustrated, but can replace performing artist A And performing artist B or performing artist C is selected, it is also possible to 1 user using 1 user terminal 40,60,70, selection includes performing artist A Multiple performing artists inside, such as performing artist B and/or performing artist C.In the case where having selected multiple performing artists, only if it were not for The image 72d for indicating the gift of the background image of live streaming publication, then both can choose different gifts, also can choose identical Gift.

According to above-mentioned live streaming delivery system 1, the effect that can be exemplified below.

(1) in live streaming delivery system 1, as user demand, such as user is for wishing that performing artist receives the heart of gift Feelings can buy gift until performing artist receives and give to performing artist.It is connect in addition, user wants raising performing artist as much as possible By a possibility that, therefore, can gift be launched near performing artist as far as possible.In addition, example can be generated as competitive spirit between user The competitive spirit as performing artist once received the gift of oneself submitting, oneself was once recompensed.Thereby, it is possible to promote user's Gift purchase.Like this, the income of network operator and performing artist be can be improved.

(2) in studio 10, user A, B, C can by make studio 10 is launched acted as gift and it is right In performing artist's A, B, C Presenting gifts that studio 10 is played.The gift given is shown in studio 10 as image 75 is fallen On ground.As a result, performing artist A, B, C also can visual identity from the users such as bean vermicelli A, B, C given gift.In addition, in user The front of performing artist A, B, C are also shown in terminal 40,60,70, therefore, user A, B, C can also be seen that the gift oneself launched Product reach the front of performing artist A, B, C.It like this, also can be as being in even if user A, B, C are not practical to be in studio 10 Studio 10 passes through 40,60,70 pairs of performing artist's A, B, C Presenting gifts of user terminal like that.In addition, performing artist A, B, C also can Carry out the movement as collecting actual present.

(3) it is made that in performing artist A, B, the C for being given away gift by the object of the gift shown on the ground of studio 10 As pick up movement in the case where, in user terminal 40,60,70, gift be shown as seem the performing artist A for being received gift, B, C is actually worn on it like that.Thus, the user of gift has been given it can be seen that the gift oneself given is received.

(4) user make to studio 10 launch such movement and to the performing artist A, B played in studio 10, When C Presenting gifts, from user terminal 40,60,70 send with arm to brandish relevant acceleration information, angle-data, angle fast Degree according to etc. operation datas.Thus, it is possible to change the gift position that gift is fallen according to speed of arm brandished etc..Cause And user A, B, C can adjust waving for arm to fall gift in front of performing artist A, B, C of selection as much as possible Dynamic speed etc., so as to improve entertainment.

(5) performing artist's A, B, C visual identity can be made to have given the User ID of user A, B, C of gift.

(6) performing artist A, B, C can user A, B, C to gift return a salute.Thereby, it is possible to performing artist A, B, C with Two-way exchange is realized between user.

(7) gift returned a salute can be also shown in user terminal 40,60,70 according to the movement of performing artist A, B, C.And And the operation for the gift returned a salute in time is picked up by being able to use user terminal 40,60,70, it can further increase Entertainment.

In addition, above-mentioned live streaming delivery system also as follows can suitably change like that.

In the case where performing artist A, B, C user A, B, C to gift return a salute, performing artist A, B, C can not also The movement returned a salute using image 77 of returning a salute.In this case, can user A, B, C to gift post the tangible gift returned a salute Object.Thereby, it is possible to simplify the processing of live streaming delivery system 1.

About the product of returning a salute, material object can also be mailed to user A, B, C in the future by tangible present.It, can in mailing With not actual signature ball, paper label and/or the albums such as commodity relevant to performing artist and/or CD, DVD, concert Discount coupon etc..In the case where mailing is returned a salute, carried out to the user for having received the intention returned a salute is registered in database 26.It should In the case of benefactor either performing artist A, B, C, are also possible to the network operator of this system.In addition, in the feelings for receiving failure Under condition, can be set to the user can not receive tangible present (not posting).

Performing artist A, B, C can not also user A, B, C to gift return a salute.That is, server 20 can be omitted back Gift processing, or the product of returning a salute can not also be posted even if having received gift.

As an example, in the case where user A, B, C not to gift return a salute, can not by User ID with give Gift foundation be associatedly managed.Thereby, it is possible to simplify the processing of live streaming delivery system 1.

In the case where having touch panel as user terminal 60, finger or stylus can be used and showing Carry out the direction of performing artist from touch control operation to display A, B, C on the display surface of live data, thus to the performing artist A of selection, B, C carries out the operation of Presenting gifts.In this case, not needing acceleration transducer and/or gyrosensor in user terminal.This Outside, in the case where user is using the user terminal for not having touch panel, mouse can be used and carry out making cursor to display The mobile operation in the direction of performing artist A, B, C to carry out performing artist A, B, C of selection the operation of Presenting gifts.

As long as the image that falls for the gift launched to performing artist A, B, C is displayed at least gift position, dropping As the track until reaching gift position can be omitted.

The practical real space performed of performing artist can be the space other than studio 10, be also possible to that meeting-place is broadcast live The concert meeting-place and/or.In this case, projector 16 shows the image of gift before the lights, user can make in auditorium Itself small-sized and portable information processing terminal as user terminal 60 carries out the operation that gift is launched to performing artist.

The image of gift is shown and is not limited to projector 16 in the method for studio 10.As an example, it may be constructed To arrange multiple panel display boards such as liquid crystal display panel in such a way that display surface is towards ground, and it is coated on the display face Transparent synthetic resin and the ground for constituting studio 10, and the image of gift is shown on the ground.In addition it is also possible to only with sharp Optical indicator indicates gift position.Also aerial display technology, air-borne imagery technology, aerial imagery (aerial can be used Imaging) technology shows gift.Gift can with two dimensional image (computer graphical (CG:computer graphics)) or 3-D image (computer graphical (CG)) is shown.Furthermore, it is also possible to by being paved with more poles on ground and making pole opposite It is gone up and down on the vertical direction in ground, and makes ground with wave change, to show the image of gift.In addition, by the object of gift Means as being shown in studio 10 are also possible to combine these devices.

The gift returned a salute can also be registered in database 26, when being broadcast live next time, by the gift returned a salute to table The person of drilling launches.Such gift is the articles not for sale gift that user can not buy.In the competing conjunction of the gift of the gift and other users, It can carry out the control for making the gift that performing artist preferentially be allowed to wear.It, can also be with as unsaleable gift either ornament It is addition special efficacy, can also be background image.

The specific movement that performing artist and/or user make (such as launches the movement of object or to which direction with what The intensity of kind degree is launched) it is not limited to the test section inspection based on test section and/or smart machine terminal 60a by smartwatch 50 The testing result measured judges (detection).Can also for example be calculated based on the image obtained by camera inter-frame difference and/ Or mobile vector, to be judged.

As an example, detected without using acceleration transducer and/or gyrosensor user A, B, C to performing artist A, B, The movement that user carries out when C Presenting gifts, and for example proceed as follows.User front setting IP Camera, take the photograph Camera etc. has the camera of dynamic image shooting function.Camera herein as an example, is and notebook PC The IP Camera being wholely set, as an example, for the IP Camera and/or video camera being connect with desktop personal computer.This It outside, as an example, is the camera for being built in smart machine terminal.Also, user terminal 40,60,70, server 20 or remove Device except this calculates the mobile data for launching image of user, and base according to the inter-frame difference for the frame for constituting video data The movement launched is detected in mobile data.Or detect user's by detecting the image mobile vector from the frame as benchmark Launch the movement of object.Then, studio 10 ground gift position and/or user terminal 40,60,70 display surface In, by performing artist A, B, C can identify gift track and/or gift in a manner of show.

In addition it is also possible to detect performing artist by using the image analysis of above-mentioned inter-frame difference and/or mobile vector A, B, C obtain the movement of user A, B, C to performing artist A, B, C gift given.As an example, it is examined by above-mentioned image analysis It surveys the movement squatted down or bent over when performing artist A, B, C pick up gift and/or touching gift or touching shows the gift position of gift The movement set.Then, performing artist A, B, C are able to carry out and wears gift or the processing to performing artist A, B, C addition special efficacy.

Furthermore, it is possible to detect the movement that performing artist A, B, C wear the gift obtained using above-mentioned image analysis etc..As An example is able to use above-mentioned image analysis to detect performing artist A, B, C movement that gift is mobile to the head of oneself.

Further, above-mentioned image analysis processing also can be used and carry out gift to detect performing artist A, B, C to user A, B, C The movement returned a salute.As an example, performing artist A, B, C can be detected using above-mentioned image analysis in studio 10.

That is, can either be handled by above-mentioned image analysis and be carried out to detect performing artist A, B, C without using depth information Movement, can also be handled by above-mentioned image analysis and detect the movement that user A, B, C are carried out.

As operation data, it is whole to can not be acceleration information, angle-data, angular velocity data, and only as At least acceleration information of mobile data.Because the flying distance for the gift launched can be calculated according to acceleration information Deng.

In studio 10, it is convenient to omit studio display 17.In this case, launching gift using the display of projector 16 ID image 76a of the User ID of the user of the image of product etc..

In the case where having given excessive gift to performing artist A, B, C, in studio, display 17 and/or user are whole In the display surface at end 40,60,70, the image of the gift of display is excessive.Similarly, on the ground of studio 10, by projector The image of the gift of 16 displays is excessive.In this case, server 20 is more than threshold value in the quantity of the user terminal of participation When, randomly select user terminal, and by the image of the gift of the user terminal from extraction show studio display 17 and/ Or in the display surface of user terminal 40,60,70.In addition, server 20 passes through projector 16 for the user terminal from extraction The image of gift is shown on the ground of studio 10.

As from user A, B, C to the gift of performing artist A, B, C present given and/or from performing artist A, B, C to The gift of returning a salute that family A, B, C are carried out has the something only " sent out " from user A, B, C to performing artist A, B, C as an example.This Outside, as an example, have from user A, B, C user A, B, C to performing artist A, B, C " giving " performing artist A, B, C are expressed thanks, Something including the emotion blessed or emerged as the times demand.Further, as an example, have user from user A, B, C for performing artist A, B, C A, the gift (property) of B, C purchase " gives " something of performing artist A, B, C.Further, as an example, have user A, B, C The gift (ownership) of purchase " gives " something of performing artist A, B, C.

As gift, it can be from user A, B, C to the gift that performing artist A, B, C give and be merely displayed in user terminal 40, the image in the display surface of 60,70 display surface and/or studio display 17.Such gift can be from (a) of Fig. 5 Shown in gift selection image 72 in select, be also possible to gift select image 72 it is unrelated, by user oneself production make The stage of performing artist becomes the gifts such as magnificent image data.It is such only in the display surface of user terminal 40,60,70 and/or The gift shown in the display surface of studio display 17 is collected the charges when user buys gift as an example.In addition, It can be in the case where homemade gift free.

The wearing product gift and/or special efficacy gift that performing artist A, B, C as hair band is worn can be real in performing artist A, B, C It collects the charges again when obtaining gift on border.I.e., it is possible to when user A, B, C buy gift and when performing artist A, B, C obtain gift It is charged twice.Alternatively, it is also possible to only charge when performing artist A, B, C obtain gift.

As returning a salute from performing artist A, B, C to user A, B, C, can be the user for only receiving and returning a salute can be watched Display and/or performance.It, can be without returning a salute from performing artist A, B, C to user A, B, C in this case, as an example Movement.Also, it under such circumstances, can also be in kind to user A, B, C mailing etc. without the ball etc. that will sign.

As gift, it is also possible to include user A, B, C image data and/or dynamic image number using software development According to summary procedure inside.As summary procedure, as an example, be move, make containing image performing artist stage become China The special efficacy program of beautiful performance etc..

Description of symbols

1... delivery system, 2... network, the studio 10..., 11... playing device, 12... loudspeaker, 13... is broadcast live Microphone, 14...RGB camera, 15... depth camera, 16... projector, the studio 17... display, 18... gift Product, 20... server, 21... audio IF, 22...RGB camera IF, 23... depth camera IF, 24... projector IF, 25... IF, 26... database, 27... data store, 28... network IF, 29... main memory, 30... control are shown Portion, 40... user terminal, 41... audio IF, 42... show that IF, 43... network IF, 44... communication IF, 45... data are deposited Storage portion, 46... operation IF, 47... main memory, 48... control unit, 49... display unit, 50... smartwatch, 51... are passed Sensor, 52... communication IF, 53... data store, 54... main memory, 55... control unit, 60... user terminal, 60a... smart machine terminal, 61... audio IF, 62... show that IF, 63... operate IF, 64... sensor, 65... network IF, 66... data store, 67... main memory, 68... control unit, 69... display unit, 70... user terminal, 71... Live views, 72... gift select image, 72a... image, 72b... image, 72c... image, 72d... image, 73... Performing artist selects image, 73a... the 1st that image, 74... performing artist is prompted to determine that image, 74a... the 2nd prompt image, 75... Fall image, 76... obtains image, 76a...ID image, 77... return a salute, and image, 78... receive image, 81... sovereign remedy Picture, 82... gift box image.

Claims (13)

1. a kind of display control program, which is characterized in that have:

Display device control unit is shown in the image of the real space where performing artist aobvious as the object of live streaming publication In showing device；

Acquisition unit obtains the three dimensional local information of the real space；

Test section, detection user are used for the user action to performing artist's Presenting gifts；With

Gift display control section, the user that the three dimensional local information and the test section obtained based on the acquisition unit is detected The user action information of movement calculates the gift position that the gift should be configured on the real space, and with the table The person of drilling, which can know, otherwise shows the calculated gift position on the real space.

2. display control program as described in claim 1, which is characterized in that

The user action is detected by the smart machine terminal that the user is kept.

3. display control program as claimed in claim 2, which is characterized in that

The user action is the movement that the user launches object.

4. display control program as claimed in claim 3, which is characterized in that

The user action information is the action data of the user.

5. display control program as described in any one of claims 1 to 4, which is characterized in that

The display device control unit is calculated should configure gift image corresponding with the gift position in the image Gift image position, and the image and the gift image is made to be shown in gift image position with the gift image Mode is shown in a display device.

6. display control program as claimed in claim 5, which is characterized in that

The display device control unit displays in real time the gift image relative to the image.

7. such as display control program described in claim 5 or 6, which is characterized in that

It is determined as that the performing artist is located at the model for the gift given from the user to the performing artist in the test section In the case where in enclosing, the display device control unit makes the gift image and performing artist foundation associatedly be shown in display In device.

8. display control program as claimed in claim 7, which is characterized in that

The user action that three dimensional local information that the range of the gift is obtained based on the acquisition unit, the test section detect Information and the type of the gift determine.

9. such as display control program according to any one of claims 1 to 8, which is characterized in that

The display device control unit is moved for giving gift of returning a salute to the user being determined as that the performing artist is made that In the case where work, to gift of returning a salute described in user delivery.

10. display control program as claimed in claim 9, which is characterized in that

The display device control unit is returned a salute when being determined as that the user has carried out receiving movement described in user delivery Gift, when being determined as that the user does not carry out receiving movement not to gift of returning a salute described in user delivery.

11. display control program as claimed in claim 10, which is characterized in that

The display device control unit makes display relevant to returning a salute be merely displayed in the user for becoming the object of the gift of returning a salute In the display device of terminal, shown in the display device of the user terminal of the user not other than it.

12. the display control program as described in any one of claim 1~11, which is characterized in that

The gift is the gift of user's purchase.

13. a kind of display control method, which is characterized in that

The image of real space where live streaming publication performing artist,

The three dimensional local information of the real space is obtained,

Detection user is used for the user action to performing artist's Presenting gifts,

Based on the user action information of the acquired three dimensional local information and the detected user action, calculate The gift position of the gift should be configured on the real space, and can be known with the performing artist and will be calculated otherwise The gift position be shown on the real space.