JP7279839B1 - REMOTE VIDEO DISTRIBUTION SYSTEM AND REMOTE VIDEO DISTRIBUTION METHOD - Google Patents

Abstract

An object of the present invention is to distribute an image captured at a remote location as an image having a color tone desired by a user. According to one aspect of the present disclosure, a video distribution device that distributes video data captured by an imaging device located at a remote location; a video reception unit that receives the video data distributed by the video distribution device; A video display unit that displays video using video data received by the video reception unit, a reception unit that receives user operations, and instruction information that instructs to change parameters of the video data based on the operations received by the reception unit. a video display device comprising: an information transmission unit for transmitting to a video distribution device, wherein the video distribution device distributes video data in which the parameter is changed based on instruction information to the video display device. distribution system. [Selection drawing] Fig. 1

Description

Application of Article 30, Paragraph 2 of the Patent Act [1] [Website] Date of posting on website: May 30, 2020 Website address: https://solution. toppan. co. jp/secure/service/remopick. html Remote communication service "RemoPick" <Reference> Website printout

The present invention relates to a remote video distribution system and a remote video distribution method.

Techniques for sharing information among a plurality of terminal devices are known. For example, a remote work support system described in Patent Document 1 displays a pointer on each touch panel of a plurality of terminal devices in order to support remote work using a plurality of terminal devices. Each of the plurality of mobile terminals has a relative coordinate system whose origin is the feature point of the image based on the relative positional relationship between the position information of the feature point of the image displayed on the display screen of the touch panel and the position information of the pointer. Coordinates are adjusted according to the difference in display screen size between other portable terminals that form and share images. This enables the remote work support system to share images and pointer information whose coordinates are correctly coordinated among a plurality of portable terminals.

JP 2014-197745 A

However, although the remote work support system can share position information among a plurality of mobile terminals, it cannot share the color tone of images among a plurality of mobile terminals. For example, when an image captured at a remote location is distributed to a terminal device owned by a user, it may not be possible to share the image with the color tone desired by the user.

The present disclosure has been made in view of such circumstances, and provides a remote video distribution system and a remote video distribution method capable of distributing video captured at a remote location as video having a color tone desired by the user. It is intended to

The present disclosure has been made to solve the above-described problems, and one aspect of the present disclosure is a video distribution device that distributes video data captured by an imaging device located at a remote location; an image receiving unit for receiving the image data received by the image receiving unit; an image display unit for displaying an image using the image data received by the image receiving unit; an accepting unit for accepting a user operation; a video display device comprising: an information transmission unit configured to transmit instruction information for instructing change of a parameter of the video data to the video distribution device based on an operation, wherein the imaging device and the video distribution device are display units. When the instruction information is transmitted from the video display device, the video distribution device displays the video data with the parameter changed based on the instruction information. a remote video distribution system , wherein the video data is distributed to a device, and the display unit displays video using the video data in which the parameters have not been changed.

Another aspect of the present disclosure is a step of generating video data by an imaging device located at a remote location, a step of a video distribution device distributing the video data captured by the imaging device, and a video display device comprising: a step of receiving the video data distributed by the video distribution device; a step of the video display device displaying video using the received video data; and transmitting instruction information to the video distribution device to instruct a change of a parameter of the video data, wherein the imaging device and the video distribution device are mounted on the head of the video distributor together with the display unit, and the instruction information is transmitted to the video distribution device. is transmitted from the video display device, the video distribution device distributes the video data in which the parameter is changed based on the instruction information to the video display device, and the display unit a remote video distribution method , wherein the video is displayed using the video data that has not been processed .

According to one aspect of the present invention, it is possible to distribute an image captured at a remote location as an image having a color tone or the like desired by the user.

1 is a block diagram showing a configuration example of a remote video distribution system according to an embodiment; FIG. 1 is a diagram illustrating an example of a video distribution device and a user terminal device according to an embodiment; FIG. 2 is a block diagram showing an example of functional configurations of a video distribution device and a user terminal device according to an embodiment; FIG. 4 is a sequence diagram showing data, information, and the like transmitted and received between the video distribution device and the user terminal device according to the embodiment; FIG.

Hereinafter, a remote video distribution system and a remote video distribution method to which the present invention is applied will be described with reference to the drawings.

[Configuration example of remote video distribution system]
FIG. 1 is a block diagram showing a configuration example of a remote video distribution system according to an embodiment. The remote video distribution system includes, for example, a user terminal device 100, a video distribution device 200, an API (Application Programming Interface) 300, and a room management device 400. The user terminal device 100, the video distribution device 200, the API 300, and the room management device 400 have communication interfaces such as NICs (Network Interface Cards) or wireless communication modules for connecting to a network NW such as the Internet. The network NW may include general networks such as the Internet and private networks such as local 5G or Wifi®. In the remote video distribution system of the embodiment, the video distribution device 200 is smart glasses worn by a video distributor who sells agricultural products such as apples on a farm, and the user terminal device 100 is operated by a user who purchases agricultural products. smartphones, tablets, personal computers, etc.

The remote video distribution system connects the user terminal device 100 and the video distribution device 200 using room information, and provides a video distribution service for transmitting and receiving video and audio in real time. In the following embodiment, a video distribution service for transmitting and receiving information such as voice and pointer while distributing video imaged by the video distribution device 200 to the user terminal device 100 in real time will be described.

The user terminal device 100 is a smartphone, a personal computer, or the like operated by a user who receives a video distribution service. In the user terminal device 100, for example, application software for receiving a video distribution service is executed by a processor such as a CPU (Central Processing Unit) stored in a program memory.

The video distribution device 200 includes, for example, a glasses-type camera device such as smart glasses worn by a video distributor. The video distribution device 200 is not limited to a device worn by a video distributor, and may be a portable terminal device such as a smart phone. The video distribution device 200 captures images of, for example, farm products, and transmits captured video data to the user terminal device 100 via the API 300 .

The API 300 is, for example, an API capable of transmitting and receiving message information, video data, and audio data in real time. The message information is, for example, information indicating pointer coordinates in video data. The API 300 transmits and receives message information between the user terminal device 100 and the video distribution device 200 in real time. The API 300 transmits and receives video data and audio data between the user terminal device 100 and the video distribution device 200 in real time.

The user terminal device 100 may transmit instruction information to the video distribution device 200 using the API 300 . The instruction information is information for instructing change of the parameters of the video data based on the user's operation. The parameters of the video data are parameters that change the color tone of the video displayed on the user terminal device 100, but are not limited to this, and are the white balance, contrast, color temperature, sharpness, redness, etc. of the video. Any parameter that adjusts the

The room management device 400 is an information processing device for providing a room for transmitting/receiving video etc. in real time to the user terminal device 100 and the video distribution device 200 . Room management device 400 includes, for example, database device 410 , authentication unit 420 , and management unit 430 . The database device 410 accepts a room information setting request transmitted from the user terminal device 100 . Database device 410 accepts a room information setting request transmitted from video distribution device 200 . Authentication unit 420 receives user information of a video distribution service from an operator who provides the video distribution service. The authentication unit 420 authenticates the user information of the video distributor and the user information of the user terminal device 100 as the user of the video distribution service, and registers the user information in the database device 410 when the authentication is successful. The management unit 430 manages entry and exit of rooms.

The room management device 400 grants the right to use the video distribution service by receiving a room reservation from the video distribution device 200, for example. The video distribution device 200 acquires, for example, room information such as the date and time of using the video distribution service, an ID, and a password, and shares it with the user terminal device 100 . Accordingly, the video distribution device 200 and the user terminal device 100 can receive the video distribution service by transmitting room information corresponding to the usage authority of the video distribution service to the room management device 400 .

FIG. 2 is a diagram showing an example of the video distribution device 200 and the user terminal device 100. As shown in FIG. The video distribution device 200 includes, for example, a main body section 200a, a housing section 200b, and mounting sections 200c and 200d. The main unit 200a accommodates a communication unit and the like for operating the video distribution device 200. As shown in FIG. The housing portion 200b accommodates a camera portion 210, a microphone portion 220, a speaker portion (not shown), and a display portion 230. As shown in FIG. Mounting parts 200c and 200d are parts for mounting video distribution device 200 on the head of a video distributor. The video distribution device 200, when worn by the video distributor, can capture an image close to the line of sight of the video distributor, and can detect voices emitted from the video distributor and sounds around the video distributor. Thereby, the video distribution device 200 transmits video data and audio data to the user terminal device 100 . Also, the video distribution device 200 can receive audio data from the user terminal device 100 and transmit the user's voice to the video distributor via the speaker unit.

The user terminal device 100 includes a display section 110, a microphone section and a speaker section (not shown). The display unit 110 functions as a touch panel by including a built-in sensor. The display unit 110 displays a video displayed using the video data received from the video distribution device 200, the video adjustment operation image 100a, and the pointer 100b. Image adjustment operation image 100a includes an image corresponding to a parameter representing how the image displayed on display unit 110 is viewed. Specifically, the image adjustment operation image 100a includes an operation image (black circle) for adjusting the contrast of the image, an operation image for adjusting the color temperature of the image, an operation image for adjusting the sharpness of the image, and the redness of the image. and a manipulation image for adjusting the The user terminal device 100 receives a user's operation of sliding each operation image in the horizontal direction, and generates instruction information that instructs parameters of video data that is the source of the video displayed on the display unit 110 . The pointer 100b is displayed on the display unit 110 based on the position touched by the user's finger. Position information indicating the position touched by the user's finger is transmitted to the video distribution device 200 . As a result, for example, when the user wants to display an image with a stronger reddish tint in order to search for an apple that looks delicious, the user can adjust the reddishness as a parameter of the video data by operating the operation image to adjust the reddishness. Instruction information can be sent from the user terminal device 100 .

The user terminal device 100 may select the optimum pointer color based on the user's operation or automatically. As a result, the user terminal device 100 can transmit the position information in a pointer color that is easy for the video distributor to understand.

[Configuration example of video distribution device 200 and user terminal device 100]
FIG. 3 is a block diagram showing an example of functional configurations of the video distribution device 200 and the user terminal device 100. As shown in FIG. The video distribution device 200 includes, for example, a camera section 210, a microphone section 220, a display section 230, a communication section 240, an image processing section 250, and a speaker section 260. The communication unit 240 performs wireless communication using, for example, Wi-Fi or Bluetooth. The communication unit 240 transmits and receives video data, audio data, and the like.

The image processing unit 250 outputs the video data generated by the camera unit 210 to the display unit 230 to display the video. When the communication unit 240 receives the pointer position information, the image processing unit 250 displays an image corresponding to the pointer at the display position in the display unit 230 corresponding to the position information. Thereby, the image processing unit 250 can share the position of the pointer displayed by the user terminal device 100 between the user and the video distributor.

The image processing unit 250 adjusts the parameters of the display data based on the instruction information received by the communication unit 240 and causes the video data after parameter adjustment to be transmitted to the user terminal device 100 via the communication unit 240 . Note that the image processing unit 250 may output either image data captured by the camera unit 210 or image data after parameter adjustment to the display unit 230 to display an image. The image processing unit 250 causes the display unit 230 to display the video data captured by the camera unit 210, thereby displaying the video with the parameters of the video data that allows the video distributor to easily check the video captured by the camera unit 210. can be made The image processing unit 250 causes the display unit 230 to display the image data after the parameter adjustment, thereby allowing the image distributor to recognize images that are easy for the user of the user terminal device 100 to recognize.

The image processing unit 250 may cause the display unit 230 to display the contents of the instruction information received from the user terminal device 100 as character information. Thereby, the image processing unit 250 can make the video distributor recognize that the parameter of the video data has been adjusted by the user.

The user terminal device 100 includes, for example, a display unit 110, a communication unit 120, an image processing unit 130, a speaker unit 140, an operation unit 150 corresponding to a touch panel, and a microphone unit 160. The communication unit 120 performs wireless communication using Wi-Fi, Bluetooth, or the like, for example. The communication unit 120 transmits and receives video data, audio data, and the like. The image processing unit 130 outputs the video data received by the communication unit 120 to the display unit 110 to display the video on the display unit 110 .

The image processing unit 130 is supplied with the parameters of the video data from the control unit 170 based on the operation on the operation unit 150 . The image processing unit 130 adjusts the parameter of the video data received by the communication unit 120 according to the adjusted parameter of the video data, and causes the display unit 110 to display the parameter-adjusted video data. Accordingly, the image processing unit 130 can adjust the video based on the user's operation.

The control unit 170 causes the video distribution apparatus 200 to transmit the parameter of the video data based on the operation on the operation unit 150 as instruction information. Accordingly, control unit 170 causes video distribution device 200 to generate video data whose parameters have been changed based on the instruction information, receives the video data after parameter adjustment, and displays video data on display unit 110 using the received video data. can be displayed.

In this embodiment, video distribution apparatus 200 adjusts the parameters of the display data based on the instruction information, but the parameters of the display data may be changed by other means. For example, the parameters of the video data may be adjusted by the API 300, and the parameter-adjusted video data may be displayed on both the video data display unit 230 and the display unit 110 of the user terminal device 100. FIG.

[Operation example of remote video distribution system]
FIG. 4 is a sequence diagram showing data and information transmitted and received between the video distribution device 200 and the user terminal device 100. As shown in FIG. The video distribution device 200 and the user terminal device 100 set a room by transmitting room information to the room management device 400 at the reserved date and time of the room (step S1). Thereby, the video distribution device 200 and the user terminal device 100 can establish a communication state for transmitting and receiving video data, audio data, etc. in real time via the API 300 .

The video distribution device 200 captures a video with the camera unit 210 and distributes the captured video data D10 to the user terminal device 100 (step S10). The user terminal device 100 receives the video data D10, displays the video, generates instruction information D20 based on the user's operation, and transmits it to the video distribution device 200 (step S20). The user terminal device 100 may transmit the instruction information D20 in real time in conjunction with the user's operation, or may transmit the instruction information D20 at the timing when the parameter adjustment of the video data is completed by the user's operation. The timing at which the parameter adjustment of the video data is completed may be, for example, the timing at which an operation to close the video adjustment operation image 100a is accepted.

The video distribution device 200 transmits video data D12 captured based on the instruction information D20 received from the user terminal device 100 to the user terminal device 100 (step S30). The user terminal device 100 performs display using the video data D12 whose parameters have been adjusted according to the instruction information (step S40). As a result, the user terminal device 100 can display, for example, an image with a user-preferred color tone on the display unit 110 .

When the user performs an operation of tapping an object included in the video (for example, part of an apple), the control unit 170 generates instruction information D22 for zooming a predetermined area centered on the tapped position. to the video distribution apparatus 200 (step S40). Upon receiving instruction information D22, video distribution apparatus 200 controls camera section 210 according to instruction information D22 (step S50). The video distribution device 200 transmits to the user terminal device 100 video data D14 obtained by causing the camera unit 210 to capture a zoomed-in video of the object corresponding to the tapped position. As a result, the user terminal device 100 can use the image data D14 to display a zoomed-in image of the tapped object on the display unit 110 in real time (step S60).

When transmitting a partially zoomed-in video to the user terminal device 100, the video distribution device 200 may display the zoomed-in video on the display unit 230, and display the zoomed-in video on the display unit 230. You may continue displaying the image|video imaged by the predetermined|prescribed angle of view without making it display.

The instruction information for the user terminal device 100 to control the camera unit 210 is not limited to information for zooming in on a part of the image, and may be other information. For example, the user terminal device 100 may transmit instruction information indicating the frame rate of the video to the video distribution device 200 in order to visually recognize the movement of the target object in the video smoothly. Accordingly, the camera unit 210 can change the frame rate of the video data based on the instruction information and generate video data with a high frame rate.

[Effect of Embodiment]
As described above, according to the remote video distribution system of the embodiment, the video distribution devices (200, 240) for distributing the video data captured by the imaging device (210) located at a remote location, and the video distribution A video receiving unit (120) for receiving video data distributed by the device, a video display unit (110) for displaying video using the video data received by the video receiving unit, and a reception unit (150) for receiving user operations. , 170), and an information transmission unit (120) for transmitting instruction information for instructing change of parameters of video data to the video distribution device based on the operation received by the reception unit (100); , and the video distribution device (200) can realize a remote video distribution system that distributes video data whose parameters have been changed based on the instruction information to the video display device (100). According to the remote video distribution system of the embodiment, for example, by transmitting instruction information for instructing parameters of video data so as to adjust the color tone to the video distribution apparatus, video captured at a remote location can be displayed in a color tone desired by the user. It can be distributed as a video such as In addition, according to the remote video distribution system of the embodiment, the video data parameter for setting the white balance is transmitted from the user viewing the video to the video distribution side. can be viewed.

According to the remote video distribution system of the embodiment, a step of generating video data by an imaging device (210) located at a remote location, a step of distributing video data captured by the imaging device, and a video display device ( 100) receives video data distributed by a video distribution device (200); a step in which a video display device displays video using the received video data; and transmitting instruction information for instructing change of parameters of the video data to the video distribution apparatus, wherein the video distribution apparatus distributes the video data whose parameters have been changed based on the instruction information to the video display apparatus. , a remote video distribution method can be realized. According to the remote video distribution system of the embodiment, for example, by transmitting instruction information for instructing parameters of video data so as to adjust the color tone to the video distribution apparatus, video captured at a remote location can be displayed in a color tone desired by the user. It can be distributed as a video such as

In the remote video distribution system according to the embodiment, by setting the parameters of the video data to be parameters for adjusting the color tone of the video displayed by the video data, the video captured at a remote location can be displayed in a color tone desired by the user. can be delivered.

In the remote video distribution system of the embodiment, a video distribution device (200) performs image processing on video data captured by an imaging device (210), and distributes the processed video data to the video distribution device. A processing unit (250) is provided, and the instruction information is information for controlling the image processing unit, and the image processing unit can adjust the color tone of video data captured by the imaging device based on the parameters. Thus, according to the remote video distribution system of the embodiment, parameters of video data can be adjusted by image processing regardless of the profile of the imaging device.

The appearance of the video differs depending on the usage environment of the user, the types of the camera unit 210 and the image processing unit 250 in the video distribution device 200, and the types of the display unit 110 and the image processing unit 130 of the user terminal device 100. FIG. If the appearance of the image differs depending on the usage environment or the like, the user may not be able to see the object with the desired color tone. Further, even if the appearance of the video is adjusted by the user's operation, the parameters of the video data transmitted from the video distribution device 200 cannot be adjusted, so the user may not be able to correct the color tone, etc. desired by the user. In contrast, according to the remote video distribution system of the embodiment, the instruction information based on the user's operation is transmitted from the user terminal device 100 to the video distribution device 200, and the parameters of the video data transmitted from the video distribution device 200 are adjusted. can do. As a result, according to the remote video distribution system of the embodiment, the video data having parameters suitable for the usage environment of the user is transmitted from the video distribution device 200 to the user terminal device 100, and the user further adjusts the color tone of the video. If desired, the image processing unit 130 can finely adjust the color tone and the like of the video.

According to the remote video distribution system of the embodiment, the instruction information is information for controlling the imaging device (210), and the parameter is information for zooming part of the video data by controlling the imaging device. By controlling at least one of the optical zoom function and the digital zoom function of the imaging device based on the operation of , the user can visually recognize a high-definition image of the zoomed portion.

According to the remote video distribution system of the embodiment, the instruction information is information for controlling the imaging device (210), and the parameter is information indicating the frame rate of the video data. By controlling the frame rate, it is possible to allow the user to visually recognize images with smooth motion.

According to the remote video distribution system of the embodiment, the imaging device (210) and the video distribution device (240) are mounted on the head of the video distributor together with the display unit (230), and the instruction information is sent to the video display device (100). , the video distribution device distributes the video data whose parameters have been changed based on the instruction information to the video display device, and the display unit displays video using the video data whose parameters have not been changed. can do. As a result, according to the remote video distribution system of the embodiment, even if the parameter of the video data is changed by the instruction information, it is possible to avoid affecting the video visually recognized by the video distributor. Further, according to the remote video distribution system of the embodiment, by displaying the information indicating the instruction information on the display unit, the video distributor can be made to recognize the video desired by the user.

[Variation]
The user terminal device 100 and the video distribution device 200 use the color tone of the video displayed by the display unit 230 of the video distribution device 200 and the color tone displayed by the display unit 110 of the user terminal device 100 based on the profile information of each device. The parameters of the video data may be adjusted to match the . The device profile information is information representing device characteristics such as the type, model, and settings of each of the user terminal device 100 and the video distribution device 200 . For example, the video distribution device 200 acquires the profile information of the display unit 110 of the user terminal device 100 after the room setting is completed. Video distribution device 200 adjusts the parameters of the video data generated by camera unit 210 so that display unit 110 displays video with the same color tone as the video displayed on display unit 230 . The user terminal device 100 displays an image on the display unit 110 using the image data after parameter adjustment, thereby allowing the user to visually recognize an image having the same color tone as the image displayed on the display unit 230.

The user terminal device 100 acquires the profile information of the camera unit 210 and adjusts the camera unit so that the display unit 110 displays an image having the same color tone as the image displayed on the display unit 230 of the image distribution device 200 . 210 to the video distribution apparatus 200 to adjust the parameter of the video data generated. The video distribution device 200 adjusts the parameters of the video data based on the instruction information and transmits the video data to the user terminal device 100 . The user terminal device 100 displays an image on the display unit 110 using the image data after parameter adjustment, thereby allowing the user to visually recognize an image having the same color tone as the image displayed on the display unit 230.

The remote video distribution system of the above-described embodiment is not limited to fruit picking such as virtual apple picking, but is also applicable to tourism fields such as virtual trips to sightseeing spots, remote classes, training, virtual experiences such as educational fields or manufacturing fields, Distribution such as live commerce, shopping agent, business negotiation, retail field, medical field or nursing care field using smart glasses worn by a person near the patient and a terminal operated by a doctor, underwater moving or flying drone and observer It can also be applied to the utilization of drones that use terminals to operate.

Although each embodiment and modifications have been described, these are only examples and are not limited to these. A part of the example may be combined with one or more other embodiments or one or more other modifications to realize one aspect of the present invention.

DESCRIPTION OF SYMBOLS 100... Video display apparatus, user terminal device, 100a... Operation image for video adjustment, 100b... Pointer, 110... Video display part, display part, 120... Communication part, video reception part, information transmission part, 130... Image processing part, DESCRIPTION OF SYMBOLS 140... Speaker part 150... Operation part, reception part 160... Microphone part 170... Reception part, control part 200... Video distribution apparatus 210... Imaging apparatus 210... Camera part 220... Microphone part 230... Display Unit 240 Communication unit 240 Video distribution device 250 Image processing unit 260 Speaker unit 400 Room management device 410 Database device 420 Authentication unit 430 Management unit

Claims (4)

a video distribution device for distributing video data captured by an imaging device located at a remote location;
a video receiving unit for receiving the video data distributed by the video distribution device, a video display unit for displaying a video using the video data received by the video receiving unit, and a receiving unit for receiving a user's operation; a video display device comprising: an information transmission unit that transmits instruction information for instructing change of a parameter of the video data to the video distribution device based on the operation received by the reception unit;
The imaging device and the video distribution device are mounted on the head of the video distributor together with the display unit,
When the instruction information is transmitted from the video display device, the video distribution device distributes video data in which the parameter is changed based on the instruction information to the video display device, and the display unit displaying an image using the image data whose parameters have not been changed;
Remote video distribution system. 2. The remote video distribution system according to claim 1, wherein said display unit displays information indicating said instruction information. generating video data by an imaging device located at a remote location;
a step in which a video distribution device distributes video data captured by the imaging device;
a step in which a video display device receives the video data distributed by the video distribution device;
a step in which the image display device displays an image using the received image data;
a step in which the video display device receives a user's operation and transmits instruction information for instructing a change of a parameter of the video data to the video distribution device;
The imaging device and the video distribution device are mounted on the head of the video distributor together with the display unit,
When the instruction information is transmitted from the video display device, the video distribution device distributes video data in which the parameter is changed based on the instruction information to the video display device, and the display unit displaying an image using the image data whose parameters have not been changed;
Remote video delivery method. 4. The remote video distribution method according to claim 3, wherein said display unit displays information indicating said instruction information.

Images