WO2023112115A1 - Communication system, information processing device, information processing method, program, and recording medium - Google Patents

Communication system, information processing device, information processing method, program, and recording medium Download PDF

Info

Publication number
WO2023112115A1
WO2023112115A1 PCT/JP2021/045947 JP2021045947W WO2023112115A1 WO 2023112115 A1 WO2023112115 A1 WO 2023112115A1 JP 2021045947 W JP2021045947 W JP 2021045947W WO 2023112115 A1 WO2023112115 A1 WO 2023112115A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
vehicle
camera
image
switching
Prior art date
Application number
PCT/JP2021/045947
Other languages
French (fr)
Japanese (ja)
Inventor
廣人 根岸
昂生 加峯
晶勅 床井
靖之 野田
Original Assignee
パイオニア株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パイオニア株式会社 filed Critical パイオニア株式会社
Priority to PCT/JP2021/045947 priority Critical patent/WO2023112115A1/en
Publication of WO2023112115A1 publication Critical patent/WO2023112115A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to a communication system, an information processing device, an information processing method, a program, and a recording medium, and for example, a communication system, an information processing device, an information processing method, a program, and a recording medium for providing an image from a mobile body to the outside. Regarding.
  • Patent Document 1 There is a communication system that communicates between an in-vehicle device installed in a vehicle and an external terminal located outside the vehicle.
  • a voice call is performed between a driver of a vehicle and an operator of a terminal outside the vehicle.
  • a system for transmitting an image to a terminal outside the vehicle and displaying the image on the terminal outside the vehicle has been disclosed (for example, Patent Literature 1).
  • an image taken outside the vehicle and an image inside the vehicle taken inside the vehicle are switched as appropriate and transmitted from the in-vehicle device to the terminal outside the vehicle.
  • Such video switching is performed by the driver operating the vehicle-mounted device.
  • the driver of the vehicle can only operate the operation unit of the on-vehicle unit when the vehicle is stopped, and cannot operate the operation unit while driving. Therefore, there is a problem that video switching cannot be performed while the vehicle is moving.
  • the present invention has been made in view of the above points, and in a communication system for transmitting video from an in-vehicle device to a terminal outside the vehicle, it is possible to switch between transmitting the video outside the vehicle and transmitting the video inside the vehicle while the vehicle is running.
  • One of the objects is to provide a communication system capable of
  • the invention according to claim 1 is a communication system for performing audio-visual communication in which audio and video in the vehicle acquired by a first device that moves with the vehicle are sequentially transmitted to a second device, wherein the vehicle a video transmission unit that transmits either one of an in-camera video that is a video of the inside of the vehicle and an out-camera video that is a video of the outside of the vehicle in the vehicle to the second device, and the video transmission The part switches the image to be transmitted to either the in-camera image or the out-camera image based on a switching request according to the operation of the second device while the vehicle is running.
  • a processing device comprising: an image acquisition unit configured to acquire an in-camera image, which is an image of the interior of the vehicle captured by the vehicle; and an out-camera image, which is an image of the exterior of the vehicle captured by the vehicle; a video transmission unit configured to transmit either one of video and the out-camera video to the second device, wherein the video transmission unit transmits data to the second device while the vehicle is running in response to an operation of the second device.
  • the image to be transmitted is switched to either the in-camera image or the out-camera image based on the switching request.
  • FIG. 1 is an overall configuration diagram of a communication system according to a first embodiment
  • FIG. FIG. 2 is a diagram showing the configuration of the front seat portion of the vehicle according to the first embodiment
  • 4A and 4B are diagrams illustrating the configuration of the terminal device 20 according to the first embodiment and a display mode when an out-camera image is displayed
  • FIG. FIG. 3 is a diagram showing the configuration of the terminal device 20 and the display mode when an in-camera image is displayed according to the first embodiment
  • 1 is a block diagram of an in-vehicle device according to Embodiment 1
  • FIG. 3 is a block diagram of a server according to Example 1
  • FIG. It is a figure which shows an example of the setting information stored in user DB.
  • FIG. 10 is a diagram showing another example of setting information stored in the user DB;
  • FIG. 1 is a block diagram of a terminal device according to Example 1;
  • FIG. 4 is a flowchart showing an audio/video transmission processing routine executed by an in-vehicle device; 4 is a flowchart showing an audio/video transmission processing routine executed by a server;
  • FIG. 1 is an overall configuration diagram of a communication system 100 according to this embodiment. As shown in FIG. 1, the communication system 100 is configured by connecting an in-vehicle device 10, a terminal device 20, and a server 30 mounted on a vehicle M as a mobile object via a network NW.
  • the in-vehicle device 10 and the server 30 are connected via the network NW, and the terminal device 20 and the server 30 are connected via the network NW.
  • the in-vehicle device 10 and the terminal device 20 are configured to be able to transmit and receive information via the server 30 .
  • the network NW is, for example, LTE (Long Term Evolution), 4G (4th Generation) or 5G (5th Generation: 5th generation mobile communication system) via a mobile communication network that communicates via a connection to a base station It is a network that
  • FIG. 2 is a diagram showing the front seat portion of the vehicle M on which the in-vehicle device 10 according to the first embodiment is mounted.
  • the in-vehicle device 10 is connected to each of an exterior camera 11, an in-vehicle camera 12, a GPS receiver 13, a microphone 14, a touch panel display 15, and a speaker 16 installed in the vehicle M, and a control unit for controlling these. including.
  • the in-vehicle device 10 is arranged, for example, in the central portion of the dashboard DB in the front seats of the vehicle M. As shown in FIG.
  • the vehicle exterior photography camera 11 as a first photography unit is an imaging device that photographs the situation outside the vehicle M.
  • the exterior camera 11 is a wide-angle camera that captures a wide area in front of the vehicle M through the windshield FG.
  • the vehicle exterior photographing camera 11 is arranged on the dashboard DB.
  • the in-vehicle photographing camera 12 as a first photographing unit is an imaging device that photographs the situation inside the vehicle M.
  • the in-vehicle photographing camera 12 is a camera that photographs the driver of the vehicle M.
  • the in-vehicle camera 12 is provided on the upper end of the windshield FG or on the ceiling near the upper end.
  • exterior photographing camera 11 and interior photographing camera 12 may be multidirectional cameras capable of photographing an image of the exterior of the vehicle M and an image of the interior of the vehicle M with a single camera as a first photographing unit. .
  • the GPS receiver 13 is a receiver that receives signals (GPS signals) from GPS (Global Positioning System) satellites.
  • GPS Global Positioning System
  • the GPS receiver 13 is arranged on the dashboard DB.
  • a microphone 14 as a first sound collecting unit receives sounds in the vehicle, for example, sounds emitted by the driver and fellow passengers of the vehicle M (hereinafter collectively referred to as passengers), and converts them into electrical signals for voice input. It is a device. In this embodiment, the microphone 14 is arranged on the dashboard DB.
  • the touch panel display 15 as the first display unit is a display device in which a display that performs screen display based on the control of the in-vehicle device 10 and a touch panel that accepts input operations from passengers of the vehicle M are combined.
  • the touch panel display 15 is arranged in the center of the dashboard DB.
  • the touch panel display 15 displays, for example, a navigation image in which the current position of the vehicle M (hereinafter also referred to as the current position) and the planned travel route (hereinafter also referred to as the travel route) are superimposed on the map.
  • the speaker 16 as the first sound output unit is an audio output device that outputs audio based on electrical signals transmitted from the outside.
  • the speaker 16 is provided on each of the two A-pillars AP.
  • the passenger of the vehicle M can make voice calls with the operator of the terminal device 20 (hereinafter also referred to as the user) via the microphone 14 and speaker 16 connected to the in-vehicle device 10 .
  • the positions of the in-vehicle device 10, the camera outside the vehicle 11, the camera inside the vehicle 12, the GPS receiver 13, the microphone 14, the touch panel display 15, and the speaker 16 in the front seat portion of the vehicle M are merely examples, and these are not the same. position.
  • the exterior camera 11 may be installed anywhere as long as it can capture the situation in front of the vehicle M, and may be installed on the upper end of the windshield FG or on the ceiling near the upper end.
  • the in-vehicle device 10 includes an "out-camera image” that is an image of the exterior of the vehicle M captured by the camera 11 outside the vehicle and an "in-camera image” that is an image of the interior of the vehicle M captured by the camera 12 photographed inside the vehicle. ” is sequentially transmitted to the server 30 .
  • the server 30 that has received the video information transmitted from the in-vehicle device 10 transmits (that is, transfers) one of the received "out-camera video" and "in-camera video” to the terminal device 20.
  • the server 30 notifies the in-vehicle device 10 of information about which image is currently being transmitted to the terminal device 20 , and the information is displayed on the touch panel display 15 of the in-vehicle device 10 .
  • the passenger of the vehicle M can perceive which of the in-camera image and the out-camera image is displayed on the terminal device 20 .
  • FIG. 3 is a diagram showing the configuration of the terminal device 20 and the display mode of the video displayed on the terminal device 20.
  • FIG. Here, a case where an out-camera image is displayed is shown as an example.
  • the terminal device 20 is a communication device including a frame F and a microphone 21, a speaker 22, and a touch panel display 23 housed in the frame F.
  • the terminal device 20 is a smart phone capable of communicating with others through communication.
  • the microphone 21 as a second sound collecting unit is a voice input device that receives voice uttered by the user of the terminal device 20 and converts it into an electric signal.
  • the microphone 21 is provided on one end side of the terminal device 20 .
  • the speaker 22 as a sound output unit or a second sound output unit is an audio output device that outputs audio based on electrical signals transmitted from the outside.
  • the speaker 22 is provided on the other end side of the terminal device 20 .
  • the first sound information indicating the sound collected by the microphone 14 as the first sound collector of the in-vehicle device 10 is output from the speaker 22 as the sound output unit by the control unit of the terminal device 20 .
  • the control unit of the terminal device 20 is a sound output control unit that controls the speaker 22 and outputs the sound collected in the vehicle M as the moving body sound.
  • the second sound information indicating the sound collected by the microphone 21 as the second sound collector is output from the speaker 16 as the first sound output section by the control section of the in-vehicle device 10 described above.
  • the touch panel display 23 as a display unit or a second display unit is a display device in which a display that displays a screen based on the control of the terminal device 20 and a touch panel that accepts an input operation from the user of the terminal device 20 are combined. be.
  • a touch panel display 23 is provided in the center of the terminal device 20 .
  • the operator of the terminal device 20 is configured to be able to make voice calls with the driver of the vehicle M in which the in-vehicle device 10 is mounted via the microphone 21 and the speaker 22 described above.
  • the terminal device 20 sequentially receives video information transmitted from the in-vehicle device 10 via the server 30 . Specifically, the terminal device 20 sequentially receives the video information of the in-camera video or the out-camera video captured in the vehicle M from the in-vehicle device 10 via the server 30 .
  • the communication system 100 is configured so that the driver of the vehicle M in which the in-vehicle device 10 is mounted and the operator of the terminal device 20 can communicate with each other. Further, in this embodiment, the communication system 100 is configured so that the image of the exterior or interior of the vehicle M captured by the exterior camera 11 can be displayed on the touch panel display 23 of the terminal device 20 in real time.
  • the communication system 100 is configured so that the in-vehicle device 10 can perform live streaming to the terminal device 20 .
  • audiovisual communication a communication mode in which voice communication is established between the in-vehicle device 10 and the terminal device 20 and a video image of the exterior or interior of the vehicle M is transmitted from the in-vehicle device 10 to the terminal device 20 in real time.
  • the in-vehicle device 10 and the terminal device 20 perform audio-visual communication via the server 30 .
  • the user using the terminal device 20 can experience a virtual passenger (hereinafter referred to as "VP") riding in the vehicle M virtually.
  • VP virtual passenger
  • the video received by the terminal device 20 is displayed on the touch panel display 23.
  • an out-camera image which is an image of the outside of the vehicle M
  • the second display area A2 displays a switching button SB for switching between the out-camera image and the in-camera image displayed in the first display area A1.
  • the user operates (tap) the switching button SB to switch the image to the in-camera image.
  • the terminal device 20 transmits a switching request, which is a signal requesting video switching, to the server 30 in response to the operation of the switching button SB.
  • a switching request which is a signal requesting video switching
  • FIG. 4 is a diagram showing the configuration of the terminal device 20 and the display mode when the in-camera video is displayed.
  • An in-camera image which is an image of the interior of the vehicle M, is displayed in the first display area A1, and a switch button SB is displayed in the second display area A2.
  • FIG. 5 is a block diagram showing the configuration of the in-vehicle device 10.
  • the control unit 17 is a processing device including a CPU (Central Processing Unit) and a RAM (Random Access Memory).
  • CPU Central Processing Unit
  • RAM Random Access Memory
  • the storage unit 18 is a storage device that stores and manages data necessary for the processing of the control unit 17.
  • the storage unit 18 is configured by a storage device such as a hard disk, flash memory, SSD (Solid State Drive), or the like.
  • the storage unit 18 stores map data.
  • the communication unit 19 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 17.
  • the communication unit 19 includes, for example, a NIC (Network Interface Card) for connecting to the network NW.
  • the communication unit 19 functions as a transmission unit that transmits, for example, audio acquired in the vehicle M, an image inside the vehicle M, an image outside the vehicle M, and the current position information of the vehicle M to the server 30 .
  • FIG. 6 is a block diagram showing the configuration of the server 30.
  • the control unit 31 is a processing device including a CPU and a RAM.
  • the communication unit 32 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 31.
  • the communication unit 32 includes, for example, a NIC for connecting to the network NW.
  • the communication unit 32 functions as a receiving unit that receives, for example, images of the exterior and interior of the vehicle M, audio acquired inside the vehicle M, and information on the current position of the vehicle M from the in-vehicle device 10 .
  • the communication unit 32 functions as a transmission unit that transmits images of the exterior and interior of the vehicle M and audio acquired inside the vehicle M to the terminal device 20, for example.
  • the storage unit 33 is a storage device that stores and manages data necessary for the processing of the control unit 31.
  • the storage unit 33 is configured by a storage device such as a hard disk, flash memory, SSD, or the like.
  • the storage unit 33 stores information of users (hereinafter referred to as registered users) who are registered as users on the terminal device side of audio-visual communication and setting information as to whether or not to permit video switching for each registered user. It has a user database 331 (hereinafter referred to as user DB 331) for storing.
  • FIG. 7 is a diagram showing an example of setting information stored in the user DB 331.
  • the user DB 331 stores, for each registered user, an initial setting as to whether the video displayed in audio-visual communication is an in-camera video or an out-camera video, and whether video switching is permitted (i.e., whether switching is permitted or not). ) are stored.
  • FIG. 8 is a diagram showing another example of setting information stored in the user DB 331.
  • the user DB 331 stores setting information as to whether only the in-camera image can be displayed as the image to be displayed, only the out-camera image can be displayed, or both can be displayed for each registered user. It is If both are set to be displayable, video switching is permitted. On the other hand, if only the in-camera video or only the out-camera video is set, the video cannot be switched.
  • the setting as to whether or not video switching is possible is set in advance by the user of the in-vehicle device (for example, the driver of the vehicle M) who performs audio-visual communication using the in-vehicle device 10 before starting the audio-visual communication.
  • the driver of the vehicle M operates the touch panel 23 of the in-vehicle device 10 to determine whether or not to permit switching of video for a specific user on the terminal device side (or to display either the in-camera video or the out-camera video).
  • input the settings for The in-vehicle device 10 transmits the input setting information to the server 30 .
  • the server 30 receives the setting information and stores it in the user DB 331 (that is, updates the setting information in the user DB 331).
  • Each functional block is formed by the control unit 31 reading and executing a program stored in the storage unit 33, for example.
  • the image information acquisition unit 34 is a functional block that acquires image information sequentially transmitted from the in-vehicle device 10 via the communication unit 32 . Specifically, the image information acquiring unit 34 acquires the in-vehicle image (in-camera image) and the outside image (out-camera image) of the vehicle M transmitted from the in-vehicle device 10 as image information.
  • the vehicle information acquisition unit 35 is a functional block that sequentially acquires the current position information that is sequentially transmitted from the in-vehicle device 10, that is, the current position information of the vehicle M.
  • the image transmission unit 36 transmits (transfers) one of the in-vehicle image and the exterior image acquired by the image information acquisition unit 34 to the terminal device 20 via the communication unit 32 .
  • the video transmitting/receiving unit 36 determines the type of video to be transmitted to the terminal device 20 (i.e., the video inside the vehicle or the video outside the vehicle). or ).
  • control unit 31 notifies the in-vehicle device 10 of information specifying whether the video transmitted to the terminal device 20 by the video transmission unit 36 is the in-camera video or the out-camera video. That is, the control unit 31 has a function as a notification control unit that causes the in-vehicle device 10 to notify the type of video being transmitted to the terminal device 20 .
  • the control unit 31 notifies, via the communication unit 32, that the image type displayed on the terminal device 20 has been switched. is notified to the in-vehicle device 10.
  • the touch panel display 15 of the in-vehicle device 10 displays information as to whether the image currently being displayed on the terminal device 20 is the in-camera image or the out-camera image. Based on this, the display of the information is also updated.
  • the switching permission/prohibition determination unit 37 is a functional block that determines whether or not it is permitted to switch the type of video transmitted by the video transmission unit 36 to the terminal device 20 in response to a distance change request from the terminal device 20 . For example, when the audio/video communication between the in-vehicle device 10 and the terminal device 20 is established, the switching permission/prohibition determination unit 37 determines whether or not to permit video switching by reading the setting information from the user DB 331 .
  • control unit 31 of the server 30 causes the touch panel display 23 of the terminal device 20 to display the switching button SB only when the switching permission determination unit 37 determines that video switching is permitted.
  • the video transmission unit 36 receives the switching request transmitted from the terminal device 20 in response to the operation of the switching button SB via the communication unit 32, and switches the video to be transmitted.
  • the switching permission determination unit 37 has determined that video switching is not permitted, the switching button SB is not displayed on the touch panel display 23 of the terminal device 20 .
  • FIG. 9 is a block diagram showing the configuration of the terminal device 20.
  • the control unit 24 is a processing device including a CPU and a RAM.
  • the communication unit 25 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 24.
  • the communication unit 25 includes, for example, a NIC for connecting to the network NW.
  • the communication unit 25 functions as a receiving unit that receives, for example, images of the exterior of the vehicle M and images of the interior of the vehicle M from the server 30 .
  • the communication unit 25 also functions as a transmission unit that transmits information corresponding to an operation input using the touch panel display 23 to the server 30 .
  • the touch panel display 23 of the terminal device 20 displays the image along with the in-camera image or the out-camera image.
  • a reception display for example, the switching button SB shown in FIGS. 3 and 4 for accepting the operation of the switching request is displayed.
  • a switching request is transmitted to the server 30 via the communication unit 25 in accordance with the user's operation of the reception display.
  • FIG. 10 is a flowchart showing an audio/video transmission processing routine REX1 executed by the control unit 17 of the in-vehicle device 10.
  • the control unit 17 determines whether or not the audio/video communication connection between the in-vehicle device 10 and the terminal device 20 has been established and the audio/video communication has started (STEP 101). When the control unit 17 determines that the audio/video communication has not started (STEP101: NO), it ends the audio/video transmission processing routine REX1.
  • control unit 17 detects the image of the exterior of the vehicle M captured by the exterior camera 11 and the image of the interior of the vehicle M captured by the in-vehicle camera 12. , and voice input via the microphone 14 are sequentially transmitted to the server 30 (STEP 102).
  • the control unit 17 determines whether or not the audio/video communication has ended (STEP 103). If it is determined that the audio/video communication has not ended (STEP 103: NO), the control unit 17 returns to STEP 102 and repeats transmission of video and audio. On the other hand, if it is determined that the audio/video communication has ended (STEP103: YES), the control section 17 stops transmitting the video and audio (STEP104), and ends the process.
  • the control unit 17 of the in-vehicle device 10 continues to transmit video and audio to the server 30 as long as the audio-visual communication between the in-vehicle device 10 and the terminal device 20 continues according to the above-described audio and video transmission processing routine REX1.
  • FIG. 11 is a flowchart showing an audio/video transmission processing routine REX2 executed by the control unit 31 of the server 30.
  • FIG. 11 is a flowchart showing an audio/video transmission processing routine REX2 executed by the control unit 31 of the server 30.
  • the control unit 31 determines whether or not the audio/video communication connection between the in-vehicle device 10 and the terminal device 20 has been established and the audio/video communication has started (STEP 201). When the control unit 31 determines that the audio/video communication has not started (STEP201: NO), it ends the audio/video transmission processing routine REX2.
  • the switching permission/inhibition determination unit 37 of the control unit 31 reads from the user DB 331 the setting information of the registered user on the terminal device side for the audio/video communication that has started, It is determined whether or not to permit video switching in response to a switching request from the terminal device 20 .
  • the control unit 31 sets whether or not to switch the video in the audio-visual communication based on the determination result of the switching permission/prohibition determination unit 37 (STEP 202).
  • the video information acquisition unit 34 of the control unit 31 starts sequentially receiving the video and audio transmitted from the in-vehicle device 10 (STEP 202).
  • the control unit 31 transfers the video and audio received in STEP 202 to the terminal device 20 (STEP 203).
  • the control unit 31 determines whether or not a video switching request has been received from the terminal device 20 via the communication unit 32 (STEP 204). If it is determined that the switching request has not been received (STEP 204 : NO), the process returns to STEP 202 to continue sequentially receiving the video and audio transmitted from the in-vehicle device 10 .
  • the video transmission section 36 of the control section 31 switches the type of video (in-camera video and out-camera video) to be transmitted to the terminal device 20 (STEP 205).
  • the control unit 31 determines whether or not the audio/video communication has ended (STEP 206). If it is determined that the audio/video communication has not ended (STEP 206: NO), the control unit 31 returns to STEP 202 and continues receiving and transferring video and audio.
  • control unit 31 stops receiving and transferring video and audio (STEP207), and ends the process.
  • the video transmitted by the server 30 to the terminal device 20 is switched between the in-camera video and the out-camera video in response to a switching request from the terminal device 20 .
  • the audio/video communication for transmitting audio and video from the in-vehicle device 10 to the terminal device 20 it is possible to switch whether to transmit the in-vehicle video or the external video while the vehicle is running.
  • whether or not to switch the video in response to a switching request from the terminal device 20 is set for each registered user, and only when the switching is set to be possible, the terminal device 20 and video switching in response to the request. According to this configuration, by presetting the type of video that can be transmitted to the terminal device 20 according to who is the other party of the audio/video communication, the privacy of the passengers of the vehicle M can be protected as necessary. becomes possible.
  • the present invention is not limited to those shown in the above examples.
  • whether or not video switching is possible is set in advance before the audio/video communication, and the setting information is stored in the user DB 331.
  • the setting information is read from the user DB 331 and sent to the switching request.
  • the case of determining whether or not to switch the video according to the setting has been described as an example. However, it may be configured such that whether or not video switching is permitted in response to a switching request from the terminal device 20 can be changed during execution of the audio/video communication.
  • the passenger of the vehicle M operates the touch panel display 15 of the in-vehicle device 10 to disable switching.
  • the in-vehicle device 10 transmits a request corresponding to the input to the server 30, and the server 30 disables video switching during the audio/video communication based on the request.
  • the in-camera picture is not displayed. It is possible to flexibly change whether to switch between
  • the in-vehicle device 10 transmits both the in-camera video and the out-camera video to the server 30, and the server 30 transfers either one to the terminal device 20 as an example.
  • a configuration may be adopted in which only one of the in-camera video and the out-camera video is transmitted to the server 30 when the in-vehicle device 10 transmits the video to the server 30 .
  • a switching request from the terminal device 20 is transmitted to the in-vehicle device 10 via the server 30, and the video to be transmitted in the in-vehicle device 10 is switched.
  • the user database is provided in the storage unit 18 of the in-vehicle device 10, and the in-vehicle device 10 sets whether or not video switching is possible.
  • the switch button as the acceptance image of the switching request is displayed on the touch panel display 23 of the terminal device 20.
  • a case of displaying is described as an example.
  • the switching button itself is displayed on the touch panel display 23 of the terminal device 20, and when a switching request is received, the server 30 rejects the switching request. may be configured.
  • n number of cameras outside the vehicle and m number of cameras inside the vehicle are provided, and in response to a switching request from the terminal device 20 side, the first to n-th out-camera images and one or more of the first to m-th in-camera images may be transmitted to the terminal device 20 .

Abstract

[Problem] To provide a communication system capable of switching, during travel of a vehicle, between transmitting outside-the-vehicle video and transmitting inside-the-vehicle video. [Solution] A communication system performs audio-visual communications for sequentially transmitting, to a second device, video audio and video acquired by a first device moving together with the vehicle, the communication system being characterized by: including a video transmission unit that transmits, to the second device, an interior camera video capturing, with respect to the vehicle, the interior of the vehicle and/or an exterior camera video capturing, with respect to the vehicle, the exterior of the vehicle; and the video transmission unit switching, during travel of the vehicle, the video that is transmitted, between the interior camera video or the exterior camera video, such switching being performed on the basis of a switch request corresponding to an operation of the second device.

Description

通信システム、情報処理装置、情報処理方法、プログラム及び記録媒体Communication system, information processing device, information processing method, program and recording medium
　本発明は、通信システム、情報処理装置、情報処理方法、プログラム及び記録媒体に関し、例えば、移動体からの映像を外部に提供するための通信システム、情報処理装置、情報処理方法、プログラム及び記憶媒体に関する。 The present invention relates to a communication system, an information processing device, an information processing method, a program, and a recording medium, and for example, a communication system, an information processing device, an information processing method, a program, and a recording medium for providing an image from a mobile body to the outside. Regarding.
　車両に搭載される車載機と当該車両の外部に位置する車外端末との間で通信を行う通信ステムがある。例えば、特許文献１には、車両の運転者と車外端末の操作者との間で音声通話を行い、当該音声通話がなされているときに、車両の前方の映像を表す映像データを車載機から車外端末に送信し、当該映像を車外端末に表示させるシステムが開示されている（例えば、特許文献１）。 There is a communication system that communicates between an in-vehicle device installed in a vehicle and an external terminal located outside the vehicle. For example, in Patent Document 1, a voice call is performed between a driver of a vehicle and an operator of a terminal outside the vehicle. A system for transmitting an image to a terminal outside the vehicle and displaying the image on the terminal outside the vehicle has been disclosed (for example, Patent Literature 1).
特開２０１６－２１３７９１号公報JP 2016-213791 A
　上記のような通信システムでは、車両の外部を撮影した車外映像と車両の内部を撮影した車内映像とが適宜切り替えられ、車載機から車外端末に送信される。このような映像の切り替えは、運転者が車載機を操作することにより行われる。しかし、車両の運転者が車載機の操作部を操作することができるのは車両が停車中のタイミングに限られ、運転中は操作することができない。したがって、車両が移動している最中は、映像の切り替えを行うことができないという問題があった。 In the communication system as described above, an image taken outside the vehicle and an image inside the vehicle taken inside the vehicle are switched as appropriate and transmitted from the in-vehicle device to the terminal outside the vehicle. Such video switching is performed by the driver operating the vehicle-mounted device. However, the driver of the vehicle can only operate the operation unit of the on-vehicle unit when the vehicle is stopped, and cannot operate the operation unit while driving. Therefore, there is a problem that video switching cannot be performed while the vehicle is moving.
　本発明は、上記した点に鑑みてなされたものであり、車載機から車外端末に映像を送信する通信システムにおいて、車外映像を送信するか車内映像を送信するかを車両の走行中に切り替えることが可能な通信システムを提供することを目的の一つとする。 The present invention has been made in view of the above points, and in a communication system for transmitting video from an in-vehicle device to a terminal outside the vehicle, it is possible to switch between transmitting the video outside the vehicle and transmitting the video inside the vehicle while the vehicle is running. One of the objects is to provide a communication system capable of
　請求項１に記載の発明は、車両と共に移動する第１装置によって取得された前記車両における音声及び映像を第２装置に逐次送信する音声映像通信を行う通信システムであって、前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像のいずれか一方を前記第２装置に送信する映像送信部を有し、前記映像送信部は、前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替えることを特徴とする。 The invention according to claim 1 is a communication system for performing audio-visual communication in which audio and video in the vehicle acquired by a first device that moves with the vehicle are sequentially transmitted to a second device, wherein the vehicle a video transmission unit that transmits either one of an in-camera video that is a video of the inside of the vehicle and an out-camera video that is a video of the outside of the vehicle in the vehicle to the second device, and the video transmission The part switches the image to be transmitted to either the in-camera image or the out-camera image based on a switching request according to the operation of the second device while the vehicle is running.
　請求項７に記載の発明は、車両において撮影された映像及び前記車両の搭乗者からの音声を前記車両と共に移動する第１装置から受信して第２装置に逐次送信する音声映像通信を行う情報処理装置であって、前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像を取得する映像取得部と、前記インカメラ映像及び前記アウトカメラ映像のうちのいずれか一方を前記第２装置に送信する映像送信部と、を有し、前記映像送信部は、前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替えることを特徴とする。 According to a seventh aspect of the present invention, there is provided information for performing audiovisual communication in which images captured in a vehicle and voices from passengers in the vehicle are received from a first device that moves with the vehicle and are sequentially transmitted to a second device. A processing device, comprising: an image acquisition unit configured to acquire an in-camera image, which is an image of the interior of the vehicle captured by the vehicle; and an out-camera image, which is an image of the exterior of the vehicle captured by the vehicle; a video transmission unit configured to transmit either one of video and the out-camera video to the second device, wherein the video transmission unit transmits data to the second device while the vehicle is running in response to an operation of the second device. The image to be transmitted is switched to either the in-camera image or the out-camera image based on the switching request.
実施例１に係る通信システムの全体構成図である。1 is an overall configuration diagram of a communication system according to a first embodiment; FIG. 実施例１に係る車両の前席部分の構成を示す図である。FIG. 2 is a diagram showing the configuration of the front seat portion of the vehicle according to the first embodiment; 実施例１に係る端末装置２０の構成及びアウトカメラ映像が表示される場合の表示態様を示す図である。4A and 4B are diagrams illustrating the configuration of the terminal device 20 according to the first embodiment and a display mode when an out-camera image is displayed; FIG. 実施例１に係る端末装置２０の構成及びインカメラ映像が表示される場合の表示態様を示す図である。FIG. 3 is a diagram showing the configuration of the terminal device 20 and the display mode when an in-camera image is displayed according to the first embodiment; 実施例１に係る車載装置のブロック図である。1 is a block diagram of an in-vehicle device according to Embodiment 1; FIG. 実施例１に係るサーバのブロック図である。3 is a block diagram of a server according to Example 1; FIG. ユーザＤＢに格納されている設定情報の一例を示す図である。It is a figure which shows an example of the setting information stored in user DB. ユーザＤＢに格納されている設定情報の他の一例を示す図である。FIG. 10 is a diagram showing another example of setting information stored in the user DB; FIG. 実施例１に係る端末装置のブロック図である。1 is a block diagram of a terminal device according to Example 1; FIG. 車載装置が実行する音声映像送信処理ルーチンを示すフローチャートである。4 is a flowchart showing an audio/video transmission processing routine executed by an in-vehicle device; サーバが実行する音声映像送信処理ルーチンを示すフローチャートである。4 is a flowchart showing an audio/video transmission processing routine executed by a server;
　以下に本発明の好適な実施例を詳細に説明する。なお、以下の実施例における説明及び添付図面においては、実質的に同一または等価な部分には同一の参照符号を付している。 Preferred embodiments of the present invention will be described in detail below. In the following description of the embodiments and the attached drawings, substantially the same or equivalent parts are denoted by the same reference numerals.
　図１は、本実施例に係る通信システム１００の全体構成図である。図１に示すように、通信システム１００は、移動体としての車両Ｍに搭載されている車載装置１０と端末装置２０とサーバ３０とが、ネットワークＮＷを介して接続されて構成されている。 FIG. 1 is an overall configuration diagram of a communication system 100 according to this embodiment. As shown in FIG. 1, the communication system 100 is configured by connecting an in-vehicle device 10, a terminal device 20, and a server 30 mounted on a vehicle M as a mobile object via a network NW.
　本実施例において、車載装置１０とサーバ３０とがネットワークＮＷを介して接続され、端末装置２０とサーバ３０とがネットワークＮＷを介して接続されている。言い換えれば、車載装置１０と端末装置２０とは、サーバ３０を介して情報の送受信が可能な構成となっている。 In this embodiment, the in-vehicle device 10 and the server 30 are connected via the network NW, and the terminal device 20 and the server 30 are connected via the network NW. In other words, the in-vehicle device 10 and the terminal device 20 are configured to be able to transmit and receive information via the server 30 .
　ネットワークＮＷは、例えば、ＬＴＥ（Long Term Evolution）、４Ｇ（4th Generation）又は５Ｇ（5th Generation：第５世代移動通信システム）等の基地局への接続を介して通信を行う移動体通信網を経由するネットワークである。 The network NW is, for example, LTE (Long Term Evolution), 4G (4th Generation) or 5G (5th Generation: 5th generation mobile communication system) via a mobile communication network that communicates via a connection to a base station It is a network that
　図２は、実施例１に係る車載装置１０が搭載されている車両Ｍの前席部分を示す図である。車載装置１０は、車両Ｍ内に設置された車外撮影カメラ１１、車内撮影カメラ１２、ＧＰＳ受信機１３、マイク１４、タッチパネルディスプレイ１５及びスピーカー１６の各々に接続されており、これらを制御する制御部を含む。車載装置１０は、例えば、車両Ｍの前席のダッシュボードＤＢ内の中央部に配置されている。 FIG. 2 is a diagram showing the front seat portion of the vehicle M on which the in-vehicle device 10 according to the first embodiment is mounted. The in-vehicle device 10 is connected to each of an exterior camera 11, an in-vehicle camera 12, a GPS receiver 13, a microphone 14, a touch panel display 15, and a speaker 16 installed in the vehicle M, and a control unit for controlling these. including. The in-vehicle device 10 is arranged, for example, in the central portion of the dashboard DB in the front seats of the vehicle M. As shown in FIG.
　第１撮影部としての車外撮影カメラ１１は、車両Ｍの外部の状況を撮影する撮像装置である。本実施例において、車外撮影カメラ１１は、フロントガラスＦＧを介して車両Ｍの前方の広い領域を撮影する広角カメラである。本実施例において、車外撮影カメラ１１は、ダッシュボードＤＢ上に配されている。 The vehicle exterior photography camera 11 as a first photography unit is an imaging device that photographs the situation outside the vehicle M. In this embodiment, the exterior camera 11 is a wide-angle camera that captures a wide area in front of the vehicle M through the windshield FG. In this embodiment, the vehicle exterior photographing camera 11 is arranged on the dashboard DB.
　第１撮影部としての車内撮影カメラ１２は、車両Ｍの内部の状況を撮影する撮像装置である。本実施例において、車内撮影カメラ１２は、車両Ｍの運転者を撮影するカメラである。本実施例において、車内撮影カメラ１２は、フロントガラスＦＧの上端または当該上端付近の天井部に設けられている。 The in-vehicle photographing camera 12 as a first photographing unit is an imaging device that photographs the situation inside the vehicle M. In this embodiment, the in-vehicle photographing camera 12 is a camera that photographs the driver of the vehicle M. As shown in FIG. In this embodiment, the in-vehicle camera 12 is provided on the upper end of the windshield FG or on the ceiling near the upper end.
　なお、上記した車外撮影カメラ１１及び車内撮影カメラ１２は、第１撮影部として１台のカメラで車両Ｍの外部の映像及び車両Ｍの内部の映像が撮影可能な多方向カメラであってもよい。 In addition, the above-described exterior photographing camera 11 and interior photographing camera 12 may be multidirectional cameras capable of photographing an image of the exterior of the vehicle M and an image of the interior of the vehicle M with a single camera as a first photographing unit. .
　ＧＰＳ受信機１３は、ＧＰＳ（Global Positioning System）衛星からの信号（ＧＰＳ信号）を受信する受信機である。本実施例において、ＧＰＳ受信機１３は、ダッシュボードＤＢ上に配されている。 The GPS receiver 13 is a receiver that receives signals (GPS signals) from GPS (Global Positioning System) satellites. In this embodiment, the GPS receiver 13 is arranged on the dashboard DB.
　第１集音部としてのマイク１４は、車内の音、例えば車両Ｍの運転者や同乗者（以下、これらをまとめて搭乗者と称する）が発した音声を受けて電気信号に変換する音声入力装置である。本実施例において、マイク１４は、ダッシュボードＤＢ上に配されている。 A microphone 14 as a first sound collecting unit receives sounds in the vehicle, for example, sounds emitted by the driver and fellow passengers of the vehicle M (hereinafter collectively referred to as passengers), and converts them into electrical signals for voice input. It is a device. In this embodiment, the microphone 14 is arranged on the dashboard DB.
　第１表示部としてのタッチパネルディスプレイ１５は、車載装置１０の制御に基づいて画面表示を行うディスプレイと、車両Ｍの搭乗者からの入力操作を受け付けるタッチパネルとが組み合わされている表示装置である。本実施例において、タッチパネルディスプレイ１５は、ダッシュボードＤＢの中央部に配されている。 The touch panel display 15 as the first display unit is a display device in which a display that performs screen display based on the control of the in-vehicle device 10 and a touch panel that accepts input operations from passengers of the vehicle M are combined. In this embodiment, the touch panel display 15 is arranged in the center of the dashboard DB.
　タッチパネルディスプレイ１５には、例えば、地図に車両Ｍの現在の位置（以後、現在位置とも称する）及び移動予定経路（以後、移動経路とも称する）が重畳されたナビゲーション画像が表示される。 The touch panel display 15 displays, for example, a navigation image in which the current position of the vehicle M (hereinafter also referred to as the current position) and the planned travel route (hereinafter also referred to as the travel route) are superimposed on the map.
　第１音出力部としてのスピーカー１６は、外部から送信された電気信号に基づいて音声を出力する音声出力装置である。本実施例において、スピーカー１６は、２つのＡピラーＡＰの各々にそれぞれ設けられている。 The speaker 16 as the first sound output unit is an audio output device that outputs audio based on electrical signals transmitted from the outside. In this embodiment, the speaker 16 is provided on each of the two A-pillars AP.
　本実施例において、車両Ｍの搭乗者は、車載装置１０に接続されているマイク１４及びスピーカー１６を介して端末装置２０の操作者（以下、ユーザとも称する）と音声通話可能である。 In this embodiment, the passenger of the vehicle M can make voice calls with the operator of the terminal device 20 (hereinafter also referred to as the user) via the microphone 14 and speaker 16 connected to the in-vehicle device 10 .
　上記した車両Ｍの前席部分における車載装置１０、車外撮影カメラ１１、車内撮影カメラ１２、ＧＰＳ受信機１３、マイク１４、タッチパネルディスプレイ１５及びスピーカー１６の各々の位置は例示に過ぎず、これらは他の位置に配されていてもよい。 The positions of the in-vehicle device 10, the camera outside the vehicle 11, the camera inside the vehicle 12, the GPS receiver 13, the microphone 14, the touch panel display 15, and the speaker 16 in the front seat portion of the vehicle M are merely examples, and these are not the same. position.
　例えば、車外撮影カメラ１１は、車両Ｍの前方の状況を撮影可能であればどこに設けられていてもよく、フロントガラスＦＧの上端または当該上端付近の天井部に設けられていてもよい。 For example, the exterior camera 11 may be installed anywhere as long as it can capture the situation in front of the vehicle M, and may be installed on the upper end of the windshield FG or on the ceiling near the upper end.
　本実施例において、車載装置１０は、車外撮影カメラ１１によって撮影された車両Ｍの外部映像である「アウトカメラ映像」及び車内撮影カメラ１２によって撮影された車両Ｍの内部映像である「インカメラ映像」の映像情報をサーバ３０に逐次送信する。 In the present embodiment, the in-vehicle device 10 includes an "out-camera image" that is an image of the exterior of the vehicle M captured by the camera 11 outside the vehicle and an "in-camera image" that is an image of the interior of the vehicle M captured by the camera 12 photographed inside the vehicle. ” is sequentially transmitted to the server 30 .
　なお、車載装置１０から送信された映像情報を受信したサーバ３０は、受信した「アウトカメラ映像」及び「インカメラ映像」のうちの一方を端末装置２０に送信（すなわち、転送）する。その際、現在どちらの映像を端末装置２０に送信しているのかについての情報がサーバ３０から車載装置１０に通知され、車載装置１０のタッチパネルディスプレイ１５に表示される。これにより、車両Ｍの搭乗者は、インカメラ映像及びアウトカメラ映像のどちらが端末装置２０に表示されているのかを覚知することができる。 Note that the server 30 that has received the video information transmitted from the in-vehicle device 10 transmits (that is, transfers) one of the received "out-camera video" and "in-camera video" to the terminal device 20. At that time, the server 30 notifies the in-vehicle device 10 of information about which image is currently being transmitted to the terminal device 20 , and the information is displayed on the touch panel display 15 of the in-vehicle device 10 . Thereby, the passenger of the vehicle M can perceive which of the in-camera image and the out-camera image is displayed on the terminal device 20 .
　図３は、端末装置２０の構成及び端末装置２０に表示される映像の表示態様を示す図である。ここでは、アウトカメラ映像が表示される場合を例として示している。 FIG. 3 is a diagram showing the configuration of the terminal device 20 and the display mode of the video displayed on the terminal device 20. FIG. Here, a case where an out-camera image is displayed is shown as an example.
　端末装置２０は、フレームＦと、当該フレームＦに収められているマイク２１、スピーカー２２及びタッチパネルディスプレイ２３とを含む通信機器である。本実施例において、端末装置２０は、通信を介して他者との通話が可能なスマートフォンである。 The terminal device 20 is a communication device including a frame F and a microphone 21, a speaker 22, and a touch panel display 23 housed in the frame F. In this embodiment, the terminal device 20 is a smart phone capable of communicating with others through communication.
　第２集音部としてのマイク２１は、端末装置２０のユーザが発した音声を受けて電気信号に変換する音声入力装置である。マイク２１は、端末装置２０の一端部側に設けられている。 The microphone 21 as a second sound collecting unit is a voice input device that receives voice uttered by the user of the terminal device 20 and converts it into an electric signal. The microphone 21 is provided on one end side of the terminal device 20 .
　音出力部又は第２音出力部としてのスピーカー２２は、外部から送信された電気信号に基づいて音声を出力する音声出力装置である。スピーカー２２は、端末装置２０の他端部側に設けられている。 The speaker 22 as a sound output unit or a second sound output unit is an audio output device that outputs audio based on electrical signals transmitted from the outside. The speaker 22 is provided on the other end side of the terminal device 20 .
　上記した車載装置１０の第１集音部としてのマイク１４によって集音された音を示す第１音情報は、端末装置２０の制御部によって音出力部としてのスピーカー２２から音出力される。言い換えれば、端末装置２０の制御部は、スピーカー２２を制御して移動体音としての車両Ｍにおいて集音された音を出力する音出力制御部である。 The first sound information indicating the sound collected by the microphone 14 as the first sound collector of the in-vehicle device 10 is output from the speaker 22 as the sound output unit by the control unit of the terminal device 20 . In other words, the control unit of the terminal device 20 is a sound output control unit that controls the speaker 22 and outputs the sound collected in the vehicle M as the moving body sound.
　第２集音部としてのマイク２１によって集音された音を示す第２音情報は、上記した車載装置１０の制御部によって第１音出力部としてのスピーカー１６から音出力される。 The second sound information indicating the sound collected by the microphone 21 as the second sound collector is output from the speaker 16 as the first sound output section by the control section of the in-vehicle device 10 described above.
　表示部又は第２表示部としてのタッチパネルディスプレイ２３は、端末装置２０の制御に基づいて画面表示を行うディスプレイと、端末装置２０のユーザからの入力操作を受け付けるタッチパネルとが組み合わされている表示装置である。タッチパネルディスプレイ２３は、端末装置２０の中央に設けられている。 The touch panel display 23 as a display unit or a second display unit is a display device in which a display that displays a screen based on the control of the terminal device 20 and a touch panel that accepts an input operation from the user of the terminal device 20 are combined. be. A touch panel display 23 is provided in the center of the terminal device 20 .
　本実施例において、端末装置２０の操作者は、上記したマイク２１及びスピーカー２２を介して車載装置１０が搭載されている車両Ｍの運転者と音声通話可能に構成されている。 In this embodiment, the operator of the terminal device 20 is configured to be able to make voice calls with the driver of the vehicle M in which the in-vehicle device 10 is mounted via the microphone 21 and the speaker 22 described above.
　本実施例において、端末装置２０は、車載装置１０から送信された映像情報を、サーバ３０を介して逐次受信する。具体的には、端末装置２０は、車両Ｍにおいて撮影されたインカメラ映像又はアウトカメラ映像の映像情報を、車載装置１０からサーバ３０を介して逐次受信する。 In this embodiment, the terminal device 20 sequentially receives video information transmitted from the in-vehicle device 10 via the server 30 . Specifically, the terminal device 20 sequentially receives the video information of the in-camera video or the out-camera video captured in the vehicle M from the in-vehicle device 10 via the server 30 .
　上記したように、本実施例において、通信システム１００は、車載装置１０を搭載した車両Ｍの運転者と端末装置２０の操作者とが互いに音声通話可能に構成されている。また、本実施例において、通信システム１００は、車外撮影カメラ１１によって撮影されている車両Ｍの外部又は内部の映像が、端末装置２０のタッチパネルディスプレイ２３にリアルタイム表示可能に構成されている。 As described above, in this embodiment, the communication system 100 is configured so that the driver of the vehicle M in which the in-vehicle device 10 is mounted and the operator of the terminal device 20 can communicate with each other. Further, in this embodiment, the communication system 100 is configured so that the image of the exterior or interior of the vehicle M captured by the exterior camera 11 can be displayed on the touch panel display 23 of the terminal device 20 in real time.
　言い換えれば、通信システム１００は、車載装置１０が端末装置２０に対してライブストリーミングを実施可能に構成されている。 In other words, the communication system 100 is configured so that the in-vehicle device 10 can perform live streaming to the terminal device 20 .
　本実施例において、車載装置１０と端末装置２０との間の音声通信を確立しつつ、車載装置１０から端末装置２０に、車両Ｍの外部又は内部の映像をリアルタイムで送信する通信形態を、ここでは音声映像通信と称する。本実施例において、車載装置１０と端末装置２０とは、サーバ３０を介して音声映像通信を行っている。この音声映像通信により、端末装置２０を使用するユーザは、車両Ｍに仮想的に同乗する仮想同乗（Virtual Passenger、以下“ＶＰ”と称する）の体験をすることができる。 In the present embodiment, a communication mode in which voice communication is established between the in-vehicle device 10 and the terminal device 20 and a video image of the exterior or interior of the vehicle M is transmitted from the in-vehicle device 10 to the terminal device 20 in real time. will be referred to as audiovisual communication. In this embodiment, the in-vehicle device 10 and the terminal device 20 perform audio-visual communication via the server 30 . Through this audio-visual communication, the user using the terminal device 20 can experience a virtual passenger (hereinafter referred to as "VP") riding in the vehicle M virtually.
　端末装置２０が受信した映像は、タッチパネルディスプレイ２３に表示される。例えば、図３に示すように、第１の表示領域Ａ１には、車両Ｍの外部を撮影した映像であるアウトカメラ映像が表示される。第２の表示領域Ａ２には、第１の表示領域Ａ１に表示する映像をアウトカメラ映像にするのかインカメラ映像にするのかを切り替えるための切替ボタンＳＢが表示される。例えば、アウトカメラ映像が表示されている状態で、映像をインカメラ映像に切り替えたい場合に、ユーザは切替ボタンＳＢを操作（タップ）する。端末装置２０は、切替ボタンＳＢの操作に応じて、映像の切り替えを要求する信号である切替要求をサーバ３０に送信する。なお、以下の説明では、映像がアウトカメラ映像であるのかインカメラ映像であるのかを単に「映像の種類」とも称する。 The video received by the terminal device 20 is displayed on the touch panel display 23. For example, as shown in FIG. 3, an out-camera image, which is an image of the outside of the vehicle M, is displayed in the first display area A1. The second display area A2 displays a switching button SB for switching between the out-camera image and the in-camera image displayed in the first display area A1. For example, in a state in which an out-camera image is displayed, the user operates (tap) the switching button SB to switch the image to the in-camera image. The terminal device 20 transmits a switching request, which is a signal requesting video switching, to the server 30 in response to the operation of the switching button SB. In the following description, whether a video is an out-camera video or an in-camera video is also simply referred to as “video type”.
　図４は、端末装置２０の構成及びインカメラ映像が表示される場合の表示態様を示す図である。第１の表示領域Ａ１には車両Ｍの内部を撮影した映像であるインカメラ映像が表示され、第２の表示領域Ａ２には切替ボタンＳＢが表示される。 FIG. 4 is a diagram showing the configuration of the terminal device 20 and the display mode when the in-camera video is displayed. An in-camera image, which is an image of the interior of the vehicle M, is displayed in the first display area A1, and a switch button SB is displayed in the second display area A2.
　次に、通信システム１００を構成する車載装置１０、サーバ３０及び端末装置２０の各々の構成について、図５～７を参照して説明する。 Next, configurations of each of the in-vehicle device 10, the server 30, and the terminal device 20 that constitute the communication system 100 will be described with reference to FIGS.
　図５は、車載装置１０の構成を示すブロック図である。制御部１７は、ＣＰＵ（Central Processing Unit）やＲＡＭ（Random Access Memory）を含む処理装置である。 FIG. 5 is a block diagram showing the configuration of the in-vehicle device 10. As shown in FIG. The control unit 17 is a processing device including a CPU (Central Processing Unit) and a RAM (Random Access Memory).
　記憶部１８は、制御部１７の処理に必要なデータを記憶管理する記憶デバイスである。記憶部１８は、例えば、ハードディスク、フラッシュメモリ、ＳＳＤ（Solid State Drive）等の記憶デバイスによって構成されている。記憶部１８は、地図データを記憶している。 The storage unit 18 is a storage device that stores and manages data necessary for the processing of the control unit 17. The storage unit 18 is configured by a storage device such as a hard disk, flash memory, SSD (Solid State Drive), or the like. The storage unit 18 stores map data.
　通信部１９は、制御部１７の指示に従って外部機器とのデータの送受信を行う通信装置である。通信部１９は、例えば、ネットワークＮＷに接続するためのＮＩＣ（Network Interface Card）を含む。通信部１９は、例えば、車両Ｍにおいて取得された音声、車両Ｍの車内映像、車外映像、及び車両Ｍの現在位置情報をサーバ３０に送信する送信部として機能する。 The communication unit 19 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 17. The communication unit 19 includes, for example, a NIC (Network Interface Card) for connecting to the network NW. The communication unit 19 functions as a transmission unit that transmits, for example, audio acquired in the vehicle M, an image inside the vehicle M, an image outside the vehicle M, and the current position information of the vehicle M to the server 30 .
　図６は、サーバ３０の構成を示すブロック図である。制御部３１は、ＣＰＵやＲＡＭを含む処理装置である。 FIG. 6 is a block diagram showing the configuration of the server 30. As shown in FIG. The control unit 31 is a processing device including a CPU and a RAM.
　通信部３２は、制御部３１の指示に従って外部機器とのデータの送受信を行う通信装置である。通信部３２は、例えば、ネットワークＮＷに接続するためのＮＩＣを含む。 The communication unit 32 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 31. The communication unit 32 includes, for example, a NIC for connecting to the network NW.
　通信部３２は、例えば、車両Ｍの外部及び内部の映像、車両Ｍの内部において取得された音声、及び車両Ｍの現在位置の情報を車載装置１０から受信する受信部として機能する。また、通信部３２は、例えば、車両Ｍの外部及び内部の映像、及び車両Ｍの内部において取得された音声を端末装置２０に送信する送信部として機能する。 The communication unit 32 functions as a receiving unit that receives, for example, images of the exterior and interior of the vehicle M, audio acquired inside the vehicle M, and information on the current position of the vehicle M from the in-vehicle device 10 . In addition, the communication unit 32 functions as a transmission unit that transmits images of the exterior and interior of the vehicle M and audio acquired inside the vehicle M to the terminal device 20, for example.
　記憶部３３は、制御部３１の処理に必要なデータを記憶管理する記憶デバイスである。記憶部３３は、例えば、ハードディスク、フラッシュメモリ、ＳＳＤ等の記憶デバイスによって構成されている。 The storage unit 33 is a storage device that stores and manages data necessary for the processing of the control unit 31. The storage unit 33 is configured by a storage device such as a hard disk, flash memory, SSD, or the like.
　記憶部３３は、音声映像通信の端末装置側のユーザとして登録されているユーザ（以下、登録ユーザと称する）の情報と各々の登録ユーザについて映像の切り替えを許可するか否かについての設定情報を格納するユーザデータベース３３１（以下、ユーザＤＢ３３１と称する）を有する。 The storage unit 33 stores information of users (hereinafter referred to as registered users) who are registered as users on the terminal device side of audio-visual communication and setting information as to whether or not to permit video switching for each registered user. It has a user database 331 (hereinafter referred to as user DB 331) for storing.
　図７は、ユーザＤＢ３３１に格納されている設定情報の一例を示す図である。ユーザＤＢ３３１には、登録ユーザの各々について、音声映像通信で表示される映像がインカメラ映像かアウトカメラ映像かについての初期設定と、映像の切り替えの可否（すなわち、切り替えが許可されているか否か）についての設定情報とが格納されている。 FIG. 7 is a diagram showing an example of setting information stored in the user DB 331. FIG. The user DB 331 stores, for each registered user, an initial setting as to whether the video displayed in audio-visual communication is an in-camera video or an out-camera video, and whether video switching is permitted (i.e., whether switching is permitted or not). ) are stored.
　また、図８は、ユーザＤＢ３３１に格納されている設定情報の他の一例を示す図である。ユーザＤＢ３３１には、登録ユーザの各々について、表示する映像としてインカメラ映像のみを表示可能とするか、アウトカメラ映像のみを表示可能とするか、いずれも表示可能とするかについての設定情報が格納されている。いずれも表示可能に設定されている場合、映像の切り替えが許可される。一方、インカメラ映像のみ又はアウトカメラ映像のみに設定されている場合、映像の切り替えは不可となる。 FIG. 8 is a diagram showing another example of setting information stored in the user DB 331. In FIG. The user DB 331 stores setting information as to whether only the in-camera image can be displayed as the image to be displayed, only the out-camera image can be displayed, or both can be displayed for each registered user. It is If both are set to be displayable, video switching is permitted. On the other hand, if only the in-camera video or only the out-camera video is set, the video cannot be switched.
　なお、映像の切り替えの可否についての設定は、車載装置１０を用いて音声映像通信を行う車載装置側のユーザ（例えば、車両Ｍの運転者）によって、音声映像通信の開始前に予め設定される。例えば、車両Ｍの運転者は、車載装置１０のタッチパネル２３を操作して、端末装置側の特定のユーザについて映像の切り替えを許可するか否か（またはインカメラ映像及びアウトカメラ映像のいずれを表示させるか）についての設定を入力する。車載装置１０は、入力された設定情報をサーバ３０に送信する。サーバ３０は、当該設定情報を受信し、ユーザＤＢ３３１に格納（すなわち、ユーザＤＢ３３１内の設定情報を更新）する。 It should be noted that the setting as to whether or not video switching is possible is set in advance by the user of the in-vehicle device (for example, the driver of the vehicle M) who performs audio-visual communication using the in-vehicle device 10 before starting the audio-visual communication. . For example, the driver of the vehicle M operates the touch panel 23 of the in-vehicle device 10 to determine whether or not to permit switching of video for a specific user on the terminal device side (or to display either the in-camera video or the out-camera video). input the settings for The in-vehicle device 10 transmits the input setting information to the server 30 . The server 30 receives the setting information and stores it in the user DB 331 (that is, updates the setting information in the user DB 331).
　次に、制御部３１の機能ブロックについて説明する。各機能ブロックは、例えば記憶部３３に格納されているプログラムを制御部３１が読み出して実行することにより形成される。 Next, the functional blocks of the control unit 31 will be explained. Each functional block is formed by the control unit 31 reading and executing a program stored in the storage unit 33, for example.
　映像情報取得部３４は、車載装置１０から逐次送信される映像情報を、通信部３２を介して取得する機能ブロックである。具体的には、映像情報取得部３４は、車載装置１０から送信された車両Ｍの車内映像（インカメラ映像）及び車外映像（アウトカメラ映像）を映像情報として取得する。 The image information acquisition unit 34 is a functional block that acquires image information sequentially transmitted from the in-vehicle device 10 via the communication unit 32 . Specifically, the image information acquiring unit 34 acquires the in-vehicle image (in-camera image) and the outside image (out-camera image) of the vehicle M transmitted from the in-vehicle device 10 as image information.
　車両情報取得部３５は、車載装置１０から逐次送信される現在位置情報、すなわち車両Ｍの現在位置情報を逐次取得する機能ブロックである。 The vehicle information acquisition unit 35 is a functional block that sequentially acquires the current position information that is sequentially transmitted from the in-vehicle device 10, that is, the current position information of the vehicle M.
　映像送信部３６は、映像情報取得部３４が取得した車内映像及び車外映像のうちの一方を、通信部３２を介して端末装置２０に送信（転送）する。 The image transmission unit 36 transmits (transfers) one of the in-vehicle image and the exterior image acquired by the image information acquisition unit 34 to the terminal device 20 via the communication unit 32 .
　また、映像送受信部３６は、映像送信部３６は、端末装置２０からの切替要求を通信部３２を介して受信した場合、端末装置２０に送信する映像の種類（すなわち、車内映像か、車外映像か）を切り替える。 In addition, when the video transmitting/receiving unit 36 receives a switching request from the terminal device 20 via the communication unit 32, the video transmitting/receiving unit 36 determines the type of video to be transmitted to the terminal device 20 (i.e., the video inside the vehicle or the video outside the vehicle). or ).
　なお、制御部３１は、映像送信部３６によって端末装置２０に送信されている映像がインカメラ映像であるのかまたはアウトカメラ映像であるのかを特定する情報を車載装置１０に通知する。すなわち、制御部３１は、端末装置２０に送信している映像の種類を車載装置１０にて告知させる告知制御部としての機能を有する。 Note that the control unit 31 notifies the in-vehicle device 10 of information specifying whether the video transmitted to the terminal device 20 by the video transmission unit 36 is the in-camera video or the out-camera video. That is, the control unit 31 has a function as a notification control unit that causes the in-vehicle device 10 to notify the type of video being transmitted to the terminal device 20 .
　制御部３１は、映像送受信部３６が端末装置２０からの切替要求に応じて映像の種類の切り替えを行った場合、端末装置２０に表示される映像の種類が切り替わったことを通信部３２を介して車載装置１０に通知する。上記の通り、車載装置１０のタッチパネルディスプレイ１５には、現在端末装置２０に表示されているのがインカメラ映像なのかアウトカメラ映像なのかについての情報が表示されており、サーバ３０からの通知に基づいて、当該情報の表示も更新される。 When the image transmitting/receiving unit 36 switches the image type in response to the switching request from the terminal device 20, the control unit 31 notifies, via the communication unit 32, that the image type displayed on the terminal device 20 has been switched. is notified to the in-vehicle device 10. As described above, the touch panel display 15 of the in-vehicle device 10 displays information as to whether the image currently being displayed on the terminal device 20 is the in-camera image or the out-camera image. Based on this, the display of the information is also updated.
　切替可否判定部３７は、端末装置２０からの距離替え要求に応じて映像送信部３６が端末装置２０に送信する映像の種類を切り替えることを許可するか否かを判定する機能ブロックである。切替可否判定部３７は、例えば車載装置１０と端末装置２０との間の音声映像通信の確立時に、ユーザＤＢ３３１から設定情報を読み出すことにより、映像の切り替えを許可するか否かを判定する。 The switching permission/prohibition determination unit 37 is a functional block that determines whether or not it is permitted to switch the type of video transmitted by the video transmission unit 36 to the terminal device 20 in response to a distance change request from the terminal device 20 . For example, when the audio/video communication between the in-vehicle device 10 and the terminal device 20 is established, the switching permission/prohibition determination unit 37 determines whether or not to permit video switching by reading the setting information from the user DB 331 .
　なお、サーバ３０の制御部３１は、映像の切り替えを許可するとの判定が切替可否判定部３７によってなされている場合にのみ、端末装置２０のタッチパネルディスプレイ２３に切替ボタンＳＢを表示させる。映像送信部３６は、切替ボタンＳＢの操作に応じて端末装置２０から送信された切替要求を通信部３２を介して受信し、送信する映像の切り替えを行う。一方、映像の切り替えを許可しないとの判定が切替可否判定部３７によってなされている場合、端末装置２０のタッチパネルディスプレイ２３には切替ボタンＳＢが表示されない。 Note that the control unit 31 of the server 30 causes the touch panel display 23 of the terminal device 20 to display the switching button SB only when the switching permission determination unit 37 determines that video switching is permitted. The video transmission unit 36 receives the switching request transmitted from the terminal device 20 in response to the operation of the switching button SB via the communication unit 32, and switches the video to be transmitted. On the other hand, if the switching permission determination unit 37 has determined that video switching is not permitted, the switching button SB is not displayed on the touch panel display 23 of the terminal device 20 .
　図９は、端末装置２０の構成を示すブロック図である。制御部２４は、ＣＰＵやＲＡＭを含む処理装置である。 FIG. 9 is a block diagram showing the configuration of the terminal device 20. As shown in FIG. The control unit 24 is a processing device including a CPU and a RAM.
　通信部２５は、制御部２４の指示に従って外部機器とのデータの送受信を行う通信装置である。通信部２５は、例えば、ネットワークＮＷに接続するためのＮＩＣを含む。 The communication unit 25 is a communication device that transmits and receives data to and from an external device according to instructions from the control unit 24. The communication unit 25 includes, for example, a NIC for connecting to the network NW.
　通信部２５は、例えば、車両Ｍの外部の映像や車両Ｍの内部の映像をサーバ３０から受信する受信部として機能する。また、通信部２５は、タッチパネルディスプレイ２３を用いた操作入力に応じた情報をサーバ３０に送信する送信部として機能する。 The communication unit 25 functions as a receiving unit that receives, for example, images of the exterior of the vehicle M and images of the interior of the vehicle M from the server 30 . The communication unit 25 also functions as a transmission unit that transmits information corresponding to an operation input using the touch panel display 23 to the server 30 .
　上記の通り、端末装置２０のユーザがインカメラ映像及びアウトカメラ映像の切り替えを許可されている登録ユーザである場合、端末装置２０のタッチパネルディスプレイ２３には、インカメラ映像又はアウトカメラ映像とともに、映像の切替要求の操作を受け付ける受付表示（例えば、図３及び図４に示す切替ボタンＳＢ）が表示される。ユーザによる受付表示の操作に応じて、切替要求が通信部２５を介してサーバ３０に送信される。 As described above, when the user of the terminal device 20 is a registered user who is permitted to switch between the in-camera image and the out-camera image, the touch panel display 23 of the terminal device 20 displays the image along with the in-camera image or the out-camera image. A reception display (for example, the switching button SB shown in FIGS. 3 and 4) for accepting the operation of the switching request is displayed. A switching request is transmitted to the server 30 via the communication unit 25 in accordance with the user's operation of the reception display.
　次に、本実施例における車載装置１０、端末装置２０及びサーバ３０の各々の具体的な動作の一例を、フローチャートを用いて説明する。 Next, an example of specific operations of each of the in-vehicle device 10, the terminal device 20, and the server 30 in this embodiment will be described using a flowchart.
　図１０は、車載装置１０の制御部１７が実行する音声映像送信処理ルーチンＲＥＸ１を示すフローチャートである。 FIG. 10 is a flowchart showing an audio/video transmission processing routine REX1 executed by the control unit 17 of the in-vehicle device 10. FIG.
　制御部１７は、車載装置１０と端末装置２０との間の音声映像通信接続が確立され、音声映像通信が開始されたか否かを判定する（ＳＴＥＰ１０１）。制御部１７は、音声映像通信が開始されていないと判定すると（ＳＴＥＰ１０１：ＮＯ）、音声映像送信処理ルーチンＲＥＸ１を終了する。 The control unit 17 determines whether or not the audio/video communication connection between the in-vehicle device 10 and the terminal device 20 has been established and the audio/video communication has started (STEP 101). When the control unit 17 determines that the audio/video communication has not started (STEP101: NO), it ends the audio/video transmission processing routine REX1.
　一方、音声映像通信が開始されたと判定すると（ＳＴＥＰ１０１：ＹＥＳ）、制御部１７は、車外撮影カメラ１１によって撮影された車両Ｍの外部の映像、車内カメラ１２によって撮影された車両Ｍの内部の映像、及びマイク１４を介して入力された音声を、サーバ３０に逐次送信する（ＳＴＥＰ１０２）。 On the other hand, if it is determined that the audio-visual communication has started (STEP 101: YES), the control unit 17 detects the image of the exterior of the vehicle M captured by the exterior camera 11 and the image of the interior of the vehicle M captured by the in-vehicle camera 12. , and voice input via the microphone 14 are sequentially transmitted to the server 30 (STEP 102).
　制御部１７は、音声映像通信が終了したか否かを判定する（ＳＴＥＰ１０３）。音声映像通信が終了していないと判定すると（ＳＴＥＰ１０３：ＮＯ）、ＳＴＥＰ１０２に戻り、制御部１７は映像及び音声の送信を繰り返し実行する。一方、音声映像通信が終了したと判定すると（ＳＴＥＰ１０３：ＹＥＳ）、制御部１７は、映像及び音声の送信を停止し（ＳＴＥＰ１０４）、処理を終了する。 The control unit 17 determines whether or not the audio/video communication has ended (STEP 103). If it is determined that the audio/video communication has not ended (STEP 103: NO), the control unit 17 returns to STEP 102 and repeats transmission of video and audio. On the other hand, if it is determined that the audio/video communication has ended (STEP103: YES), the control section 17 stops transmitting the video and audio (STEP104), and ends the process.
　車載装置１０の制御部１７は、上記した音声映像送信処理ルーチンＲＥＸ１によって、車載装置１０と端末装置２０との音声映像通信が継続する限り、映像及び音声をサーバ３０に送信し続ける。 The control unit 17 of the in-vehicle device 10 continues to transmit video and audio to the server 30 as long as the audio-visual communication between the in-vehicle device 10 and the terminal device 20 continues according to the above-described audio and video transmission processing routine REX1.
　図１１は、サーバ３０の制御部３１が実行する音声映像送信処理ルーチンＲＥＸ２を示すフローチャートである。 FIG. 11 is a flowchart showing an audio/video transmission processing routine REX2 executed by the control unit 31 of the server 30. FIG.
　制御部３１は、車載装置１０と端末装置２０との間の音声映像通信接続が確立され、音声映像通信が開始されたか否かを判定する（ＳＴＥＰ２０１）。制御部３１は、音声映像通信が開始されていないと判定すると（ＳＴＥＰ２０１：ＮＯ）、音声映像送信処理ルーチンＲＥＸ２を終了する。 The control unit 31 determines whether or not the audio/video communication connection between the in-vehicle device 10 and the terminal device 20 has been established and the audio/video communication has started (STEP 201). When the control unit 31 determines that the audio/video communication has not started (STEP201: NO), it ends the audio/video transmission processing routine REX2.
　一方、音声映像通信が開始されたと判定すると（ＳＴＥＰ２０１：ＹＥＳ）、制御部３１の切替可否判定部３７は、開始した音声映像通信についての端末装置側の登録ユーザの設定情報をユーザＤＢ３３１から読み出し、端末装置２０からの切替要求に応じた映像の切り替えを許可するか否かを判定する。制御部３１は、切替可否判定部３７の判定結果に基づいて、音声映像通信における映像の切り替えの可否を設定する（ＳＴＥＰ２０２）。 On the other hand, when it is determined that the audio/video communication has started (STEP 201: YES), the switching permission/inhibition determination unit 37 of the control unit 31 reads from the user DB 331 the setting information of the registered user on the terminal device side for the audio/video communication that has started, It is determined whether or not to permit video switching in response to a switching request from the terminal device 20 . The control unit 31 sets whether or not to switch the video in the audio-visual communication based on the determination result of the switching permission/prohibition determination unit 37 (STEP 202).
　制御部３１の映像情報取得部３４は、車載装置１０から送信された映像及び音声の逐次受信を開始する（ＳＴＥＰ２０２）。 The video information acquisition unit 34 of the control unit 31 starts sequentially receiving the video and audio transmitted from the in-vehicle device 10 (STEP 202).
　制御部３１は、ＳＴＥＰ２０２で受信した映像及び音声を、端末装置２０に転送する（ＳＴＥＰ２０３）。 The control unit 31 transfers the video and audio received in STEP 202 to the terminal device 20 (STEP 203).
　制御部３１は、通信部３２を介して端末装置２０から映像の切替要求を受信したか否かを判定する（ＳＴＥＰ２０４）。切替要求を受信していないと判定すると（ＳＴＥＰ２０４：ＮＯ）、ＳＴＥＰ２０２に戻って、車載装置１０から送信された映像及び音声の逐次受信を継続する。 The control unit 31 determines whether or not a video switching request has been received from the terminal device 20 via the communication unit 32 (STEP 204). If it is determined that the switching request has not been received (STEP 204 : NO), the process returns to STEP 202 to continue sequentially receiving the video and audio transmitted from the in-vehicle device 10 .
　一方、切替要求を受信したと判定すると（ＳＴＥＰ２０４：ＹＥＳ）、制御部３１の映像送信部３６は、端末装置２０に送信する映像の種類（インカメラ映像及びアウトカメラ映像）を切り替える（ＳＴＥＰ２０５）。 On the other hand, when it is determined that a switching request has been received (STEP 204: YES), the video transmission section 36 of the control section 31 switches the type of video (in-camera video and out-camera video) to be transmitted to the terminal device 20 (STEP 205).
　制御部３１は、音声映像通信が終了したか否かを判定する（ＳＴＥＰ２０６）。音声映像通信が終了していないと判定すると（ＳＴＥＰ２０６：ＮＯ）、ＳＴＥＰ２０２に戻り、制御部３１は、映像及び音声の受信、転送を継続する。 The control unit 31 determines whether or not the audio/video communication has ended (STEP 206). If it is determined that the audio/video communication has not ended (STEP 206: NO), the control unit 31 returns to STEP 202 and continues receiving and transferring video and audio.
　一方、音声映像通信が終了したと判定すると（ＳＴＥＰ２０６：ＹＥＳ）、制御部３１は、映像及び音声の受信、転送を停止し（ＳＴＥＰ２０７）、処理を終了する。 On the other hand, when it is determined that the audio/video communication has ended (STEP206: YES), the control unit 31 stops receiving and transferring video and audio (STEP207), and ends the process.
　以上のように、本実施例の通信システム１００では、端末装置２０からの切替要求に応じて、サーバ３０が端末装置２０に送信する映像をインカメラ映像及びアウトカメラ映像のいずれかに切り替える。かかる構成によれば、車載装置１０から端末装置２０に音声及び映像を送信する音声映像通信において、車内映像及び車外映像のいずれを送信するかを車両の走行中に切り替えることが可能となる。 As described above, in the communication system 100 of the present embodiment, the video transmitted by the server 30 to the terminal device 20 is switched between the in-camera video and the out-camera video in response to a switching request from the terminal device 20 . According to this configuration, in the audio/video communication for transmitting audio and video from the in-vehicle device 10 to the terminal device 20, it is possible to switch whether to transmit the in-vehicle video or the external video while the vehicle is running.
　また、本実施例の通信システム１００では、端末装置２０からの切替要求に応じて映像の切り替えを行うか否かが登録ユーザ毎に設定され、切り替え可能に設定されている場合にのみ、端末装置２０からの切替要求及びこれに応じた映像の切り替えが可能に構成されている。かかる構成によれば、音声映像通信の相手方が誰であるのかに応じて端末装置２０に送信可能な映像の種類を予め設定することにより、必要に応じて車両Ｍの搭乗者のプライバシーを保護することが可能となる。 Further, in the communication system 100 of the present embodiment, whether or not to switch the video in response to a switching request from the terminal device 20 is set for each registered user, and only when the switching is set to be possible, the terminal device 20 and video switching in response to the request. According to this configuration, by presetting the type of video that can be transmitted to the terminal device 20 according to who is the other party of the audio/video communication, the privacy of the passengers of the vehicle M can be protected as necessary. becomes possible.
　なお、本発明は上記実施例で示したものに限られない。例えば、上記実施例では、映像の切り替えの可否を音声映像通信の前に予め設定して設定情報をユーザＤＢ３３１に格納し、音声映像通信の開始時にその設定情報をユーザＤＢ３３１から読み出して切替要求に応じた映像の切り替えの可否を決定する場合を例として説明した。しかし、端末装置２０からの切替要求に応じた映像の切り替えの可否を、音声映像通信の実行中に変更可能に構成されていてもよい。例えば、映像の切り替えが可能な状態で音声映像通信を開始後に、車載装置側で何らかの状況の変化が起こった場合、車両Ｍの搭乗者が車載装置１０のタッチパネルディスプレイ１５を操作して切り替えを不可とするように要求する入力を行う。車載装置１０は、当該入力に応じた要求をサーバ３０に送信し、サーバ３０は当該要求に基づいて音声映像通信の途中から映像の切り替えを不可とする。かかる構成によれば、例えば車両Ｍの走行中に途中から同乗者が加わり且つその同乗者が顔を映したくない等の状況が生じた場合に、インカメラ映像を表示させないようにする等、映像の切り替えの可否を柔軟に変更することが可能となる。 The present invention is not limited to those shown in the above examples. For example, in the above-described embodiment, whether or not video switching is possible is set in advance before the audio/video communication, and the setting information is stored in the user DB 331. At the start of the audio/video communication, the setting information is read from the user DB 331 and sent to the switching request. The case of determining whether or not to switch the video according to the setting has been described as an example. However, it may be configured such that whether or not video switching is permitted in response to a switching request from the terminal device 20 can be changed during execution of the audio/video communication. For example, after starting audio-visual communication in a state in which video switching is possible, if some situation changes on the in-vehicle device side, the passenger of the vehicle M operates the touch panel display 15 of the in-vehicle device 10 to disable switching. Make an input requesting that The in-vehicle device 10 transmits a request corresponding to the input to the server 30, and the server 30 disables video switching during the audio/video communication based on the request. According to this configuration, for example, when a fellow passenger joins in the middle of traveling of the vehicle M and the fellow passenger does not want to show his face, the in-camera picture is not displayed. It is possible to flexibly change whether to switch between
　また、上記実施例では、車載装置１０がインカメラ映像及びアウトカメラ映像の両方をサーバ３０に送信し、サーバ３０がいずれか一方を端末装置２０に転送する場合を例として説明した。しかし、車載装置１０がサーバ３０に映像を送信する段階で、インカメラ映像及びアウトカメラ映像のいずれか一方のみをサーバ３０に送信する構成としてもよい。この場合、端末装置２０からの切替要求はサーバ３０を介して車載装置１０に送信され、車載装置１０において送信する映像の切り替えを行う構成となる。また、この場合、ユーザデータベースは車載装置１０の記憶部１８に設けられ、映像の切り替えの可否についての設定を車載装置１０において行う。 In addition, in the above embodiment, the in-vehicle device 10 transmits both the in-camera video and the out-camera video to the server 30, and the server 30 transfers either one to the terminal device 20 as an example. However, a configuration may be adopted in which only one of the in-camera video and the out-camera video is transmitted to the server 30 when the in-vehicle device 10 transmits the video to the server 30 . In this case, a switching request from the terminal device 20 is transmitted to the in-vehicle device 10 via the server 30, and the video to be transmitted in the in-vehicle device 10 is switched. Further, in this case, the user database is provided in the storage unit 18 of the in-vehicle device 10, and the in-vehicle device 10 sets whether or not video switching is possible.
　また、上記実施例では、ユーザＤＢ３３１に格納されている設定情報に基づいて、ユーザに映像の切り替えが許可されている場合にのみ端末装置２０のタッチパネルディスプレイ２３に切替要求の受付画像としての切替ボタンを表示させる場合を例として説明した。しかし、これとは異なり、映像の切り替えが許可されていない場合であっても切替ボタン自体は端末装置２０のタッチパネルディスプレイ２３に表示させ、切替要求が来た際にサーバ３０において切替要求を拒否する構成としてもよい。 Further, in the above-described embodiment, based on the setting information stored in the user DB 331, only when the user is permitted to switch the video, the switch button as the acceptance image of the switching request is displayed on the touch panel display 23 of the terminal device 20. A case of displaying is described as an example. However, unlike this, even if video switching is not permitted, the switching button itself is displayed on the touch panel display 23 of the terminal device 20, and when a switching request is received, the server 30 rejects the switching request. may be configured.
　また、上記実施例では、図２に示すように車外撮影カメラ１１及び車内撮影カメラ１２が一台ずつ設けられ、アウトカメラ映像及びインカメラ映像のうちのいずれか一方を端末装置２０に送信する場合を例として説明した。しかし、車外撮影カメラ及び車内撮影カメラはそれぞれ複数設けられていてもよい。例えば、ｎ個の車外撮影カメラ及びｍ個の車内撮影カメラ（ｍ、ｎは２以上の整数）が設けられ、端末装置２０側からの切替要求に応じて、第１～第ｎのアウトカメラ映像及び第１～第ｍのインカメラ映像のうちの１又は複数を端末装置２０に送信するように構成されていてもよい。 Further, in the above-described embodiment, as shown in FIG. was described as an example. However, a plurality of cameras outside the vehicle and cameras inside the vehicle may be provided. For example, n number of cameras outside the vehicle and m number of cameras inside the vehicle (m and n are integers of 2 or more) are provided, and in response to a switching request from the terminal device 20 side, the first to n-th out-camera images and one or more of the first to m-th in-camera images may be transmitted to the terminal device 20 .
１００　通信システム
１０　車載装置
１１　車外撮影カメラ
１２　車内撮影カメラ
１３　ＧＰＳ受信機
１４　マイク
１５　タッチパネルディスプレイ
１６　スピーカー
１７　制御部
１８　記憶部
１９　通信部
２０　端末装置
２１　マイク
２２　スピーカー
２３　タッチパネルディスプレイ
２４　制御部
２５　通信部
３０　サーバ
３１　制御部
３２　通信部
３３　記憶部
３３１　ユーザＤＢ
３４　映像情報取得部
３５　車両情報取得部
３６　映像送信部
３７　切替可否判定部
 
  100 communication system 10 in-vehicle device 11 vehicle exterior camera 12 vehicle interior camera 13 GPS receiver 14 microphone 15 touch panel display 16 speaker 17 control unit 18 storage unit 19 communication unit 20 terminal device 21 microphone 22 speaker 23 touch panel display 24 control unit 25 communication unit 30 server 31 control unit 32 communication unit 33 storage unit 331 user DB
34 video information acquisition unit 35 vehicle information acquisition unit 36 video transmission unit 37 switching availability determination unit

Claims (10)

  1. 　車両と共に移動する第１装置によって取得された前記車両における音声及び映像を第２装置に逐次送信する音声映像通信を行う通信システムであって、
    　前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像のいずれか一方を前記第２装置に送信する映像送信部を有し、
    　前記映像送信部は、前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替える映像切り替えを行うことを特徴とする通信システム。     A communication system for performing audiovisual communication in which audio and video in the vehicle acquired by a first device moving with the vehicle are sequentially transmitted to a second device,
    a video transmission unit that transmits either one of an in-camera video that is a video of the inside of the vehicle taken in the vehicle and an out-camera video that is a video of the outside of the vehicle in the vehicle to the second device. death,
    The video transmission unit switches video to be transmitted to either the in-camera video or the out-camera video based on a switching request according to the operation of the second device while the vehicle is running. A communication system characterized by:
  2. 　前記映像送信部に前記第２装置の操作に応じた前記映像切り替えを許可するか否かを判定する判定部を含み、前記映像送信部は、前記判定部が許可した場合にのみ前記映像切り替えを行うことを特徴とする請求項１に記載の通信システム。 The image transmission unit includes a determination unit that determines whether or not to permit the image switching according to the operation of the second device, and the image transmission unit performs the image switching only when the determination unit permits it. 2. The communication system according to claim 1, wherein:
  3. 　前記判定部は、前記第２の装置のユーザが誰かに基づいて前記映像切り替えを許可するか否かを判定することを特徴とする請求項２に記載の通信システム。 The communication system according to claim 2, wherein the determination unit determines whether or not to permit the video switching based on who the user of the second device is.
  4. 　前記判定部は、前記第１の装置のユーザから特定の操作があったか否かに基づいて前記映像切り替えを許可するか否かを判定することを特徴とする請求項２または３に記載の通信システム。 4. The communication system according to claim 2, wherein the determination unit determines whether or not to permit the video switching based on whether or not a specific operation has been performed by the user of the first device. .
  5. 　前記判定部が前記映像切り替えを許可すると判定した際に、前記第２装置において前記映像の切り替え操作を受け付ける操作受付表示を表示させることを特徴とする請求項２乃至４のいずれか１に記載の通信システム。 5. The method according to any one of claims 2 to 4, wherein when the determination unit determines that the video switching is permitted, the second device displays an operation acceptance display for accepting the switching operation of the video. Communications system.
  6. 　前記映像送信部によって送信されている映像が前記インカメラ映像であるのかまたは前記アウトカメラ映像であるのかを特定する情報を前記第１装置にて告知させる告知制御部を含むことを特徴とする請求項１乃至５のいずれか１に記載の通信システム。 A notification control unit that causes the first device to notify information specifying whether the image transmitted by the image transmission unit is the in-camera image or the out-camera image. Item 6. The communication system according to any one of Items 1 to 5.
  7. 　車両において撮影された映像及び前記車両の搭乗者からの音声を前記車両と共に移動する第１装置から受信して第２装置に逐次送信する音声映像通信を行う情報処理装置であって、
    　前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像を取得する映像取得部と、
    　前記インカメラ映像及び前記アウトカメラ映像のうちのいずれか一方を前記第２装置に送信する映像送信部と、
    　を有し、　
    　前記映像送信部は、前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替えることを特徴とする情報処理装置。     An information processing device that performs audio-visual communication for receiving video captured in a vehicle and audio from a passenger in the vehicle from a first device that moves with the vehicle and sequentially transmitting the video to a second device,
    an image acquisition unit that acquires an in-camera image that is an image of the interior of the vehicle captured in the vehicle and an out-camera image that is an image of the exterior of the vehicle captured in the vehicle;
    a video transmission unit that transmits one of the in-camera video and the out-camera video to the second device;
    has
    The image transmission unit switches the image to be transmitted to either the in-camera image or the out-camera image based on a switching request according to the operation of the second device while the vehicle is running. information processing equipment.
  8. 　車両において撮影された映像及び前記車両の搭乗者からの音声を前記車両と共に移動する第１装置から受信して第２装置に逐次送信する音声映像通信を行う情報処理装置において実行される情報処理方法であって、
    　前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像を取得する映像情報取得ステップと、
    　前記インカメラ映像及び前記アウトカメラ映像のうちのいずれか一方を前記第２装置に送信する映像送信ステップと、
    　前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替える映像切替ステップと、
    　を有することを特徴とする情報処理方法。     An information processing method executed in an information processing apparatus that performs audiovisual communication in which images captured in a vehicle and voices from passengers in the vehicle are received from a first apparatus moving with the vehicle and sequentially transmitted to a second apparatus. and
    A video information acquisition step of acquiring an in-camera video that is a video of the inside of the vehicle taken in the vehicle and an out-camera video that is a video of the outside of the vehicle taken in the vehicle;
    a video transmission step of transmitting one of the in-camera video and the out-camera video to the second device;
    a video switching step of switching video to be transmitted to either the in-camera video or the out-camera video based on a switching request according to the operation of the second device while the vehicle is running;
    An information processing method characterized by having
  9. 　車両において撮影された映像及び前記車両の搭乗者からの音声を前記車両と共に移動する第１装置から受信して第２装置に逐次送信する音声映像通信を行う情報処理装置に、
    　前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像を取得する映像情報取得ステップと、
    　前記インカメラ映像及び前記アウトカメラ映像のうちのいずれか一方を前記第２装置に送信する映像送信ステップと、
    　前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替える映像切替ステップと、
    　を実行させることを特徴とするプログラム。     An information processing device that performs audio-visual communication for receiving video captured in a vehicle and audio from a passenger in the vehicle from a first device that moves with the vehicle and sequentially transmitting the video to a second device,
    A video information acquisition step of acquiring an in-camera video that is a video of the inside of the vehicle taken in the vehicle and an out-camera video that is a video of the outside of the vehicle taken in the vehicle;
    a video transmission step of transmitting one of the in-camera video and the out-camera video to the second device;
    a video switching step of switching video to be transmitted to either the in-camera video or the out-camera video based on a switching request according to the operation of the second device while the vehicle is running;
    A program characterized by causing the execution of
  10. 　車両において撮影された映像及び前記車両の搭乗者からの音声を前記車両と共に移動する第１装置から受信して第２装置に逐次送信する音声映像通信を行う情報処理装置に、
    　前記車両において前記車両の内部を撮影した映像であるインカメラ映像及び前記車両において前記車両の外部を撮影した映像であるアウトカメラ映像を取得する映像情報取得ステップと、
    　前記インカメラ映像及び前記アウトカメラ映像のうちのいずれか一方を前記第２装置に送信する映像送信ステップと、
    　前記車両の走行中に、前記第２装置の操作に応じた切替要求に基づいて、送信する映像を前記インカメラ映像または前記アウトカメラ映像のいずれかに切り替える映像切替ステップと、
    　を実行させるプログラムを記録する記録媒体。
     
     
          An information processing device that performs audio-visual communication for receiving video captured in a vehicle and audio from a passenger in the vehicle from a first device that moves with the vehicle and sequentially transmitting the video to a second device,
    A video information acquisition step of acquiring an in-camera video that is a video of the inside of the vehicle taken in the vehicle and an out-camera video that is a video of the outside of the vehicle taken in the vehicle;
    a video transmission step of transmitting one of the in-camera video and the out-camera video to the second device;
    a video switching step of switching video to be transmitted to either the in-camera video or the out-camera video based on a switching request according to the operation of the second device while the vehicle is running;
    A recording medium that records a program that causes a program to be executed.
PCT/JP2021/045947 2021-12-14 2021-12-14 Communication system, information processing device, information processing method, program, and recording medium WO2023112115A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/045947 WO2023112115A1 (en) 2021-12-14 2021-12-14 Communication system, information processing device, information processing method, program, and recording medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/045947 WO2023112115A1 (en) 2021-12-14 2021-12-14 Communication system, information processing device, information processing method, program, and recording medium

Publications (1)

Publication Number Publication Date
WO2023112115A1 true WO2023112115A1 (en) 2023-06-22

Family

ID=86774066

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/045947 WO2023112115A1 (en) 2021-12-14 2021-12-14 Communication system, information processing device, information processing method, program, and recording medium

Country Status (1)

Country Link
WO (1) WO2023112115A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011199722A (en) * 2010-03-23 2011-10-06 Secom Co Ltd On-vehicle monitoring device
JP2011205373A (en) * 2010-03-25 2011-10-13 Secom Co Ltd On-board image monitoring device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011199722A (en) * 2010-03-23 2011-10-06 Secom Co Ltd On-vehicle monitoring device
JP2011205373A (en) * 2010-03-25 2011-10-13 Secom Co Ltd On-board image monitoring device

Similar Documents

Publication Publication Date Title
KR20080103370A (en) Real time traffic information system and its control method based on p2p service
CN111301284B (en) In-vehicle device, program, and vehicle
CN110316113B (en) Vehicle-mounted multimedia terminal, communication method thereof, vehicle-mounted multimedia communication system and vehicle
JP2003121162A (en) Communication apparatus among many parties
CN112954648B (en) Interaction method, terminal and system of mobile terminal and vehicle-mounted terminal
WO2023112115A1 (en) Communication system, information processing device, information processing method, program, and recording medium
JP2007074081A (en) On-vehicle communication apparatus
WO2012039022A1 (en) Information communicating apparatus, information communicating method, information communicating program, and information communicating system
JP2004064418A (en) Wireless communication system, cellular phone, and executable program by cellular phone
JP4076747B2 (en) Videophone system and videophone connection method
JP2020093622A (en) Control system, server, in-vehicle control apparatus, vehicle, and control method
JP3736732B2 (en) Vehicle navigation device and computer-readable storage medium
WO2020137398A1 (en) Operation control device, imaging device, and operation control method
JP2023094982A (en) Communication system, information processing device, information processing method, program, and recording medium
WO2022208656A1 (en) Information processing device, information processing method, program, and recording medium
JP7447399B2 (en) Video distribution control device, video distribution control method, and program
WO2023112113A1 (en) Communication system, information processing device, information processing method, program, and recording medium
WO2023112114A1 (en) Communication system, information processing device, information processing method, program, and recording medium
JP2018076035A (en) Conversation support device for vehicle
JP2006031583A (en) On-vehicle system and remote observation system
JP4586442B2 (en) Vehicle guidance system, vehicle guidance method, and in-vehicle device
WO2022208808A1 (en) Video transmission apparatus, video transmission method, program, and storage medium
JP2022153972A (en) Information processing device, information processing method, program, and storage medium
JP2001328499A (en) Terminal equipment for emergency information system and emergency information system
JP2022153971A (en) Information processing device, information processing method, program, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21968036

Country of ref document: EP

Kind code of ref document: A1