JP2018198074A - Communication device, control method thereof, and program - Google Patents

Abstract

To perform screen sharing between devices intended by a user.SOLUTION: A communication device that performs screen sharing with another communication device via a wireless network captures an image displayed on a display unit to generate a shared image; obtains a confirmation image allowing identification of the communication device; and transmits the shared image or confirmation image to the other communication device. The communication device transmits the confirmation image after wireless connection is established; and after transmitting the confirmation image, transmits, if it is determined by a user operation to perform screen sharing, the shared image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a communication device, a control method thereof, and a program, and more particularly to a technique suitable for use in screen sharing by transmitting an image displayed on a display to another device via a network.

  In recent years, a technique for wirelessly mirroring a display screen has been proposed in Wi-Fi CERTIFIED Miracast (registered trademark) or the like (Non-Patent Document 1). Mirroring is a technique for sharing a screen by transmitting a display screen of a transmission device to a reception device via a network. Non-Patent Document 1 proposes Wi-Fi Direct (registered trademark) as a method for directly connecting a transmission device and a reception device via a wireless network. Further, Wi-Fi Protected Setup (registered trademark) has been proposed as a method for simplifying security settings when connected to a wireless network. In WPS, there are a PIN method for setting with a password, a PBC method for setting with a push button, and an NFC method for setting using near field communication (NFC). As described above, the Wi-Fi CERTIFIED Miracast (registered trademark) allows the user to share the screen with a simple operation via the wireless network directly.

  Patent Document 1 and Patent Document 2 propose a technique for authenticating a network connection by a receiving apparatus displaying an authentication password and inputting the password displayed on the receiving apparatus from the transmitting apparatus.

“Wi-Fi CERTIFIED Miracast | Wi-Fi Alliance”, http://www.wi-fi.org/discover-wi-fi/wi-fi-certified-miracast

JP 2011-118243 A JP2003-069923

  Conventionally, when a transmission apparatus connects to a wireless network to perform screen sharing, the transmission apparatus displays a result of searching for the reception apparatus, and the user selects the reception apparatus from the search result to perform screen sharing. Had decided. Some receiving apparatuses establish a wireless network connection with a transmitting apparatus without user operation. When a transmitting device shares a screen with such a receiving device via a wireless network, the user cannot confirm in advance on the screen of the transmitting device whether the image is transmitted to the receiving device selected by the user. There was a problem. For this reason, if the user mistakenly selects a receiving device for screen sharing, the shared image may be transmitted to a receiving device that the user does not want.

  In Patent Document 1 and Patent Document 2, there has been no proposal for a method for confirming whether a screen transmission destination device is correct when a wireless network connection is established using an authentication method that does not use a password.

  The present invention has been made in view of the above problems, and an object thereof is to perform screen sharing between devices intended by a user.

  As a means for achieving the above object, a communication apparatus of the present invention comprises the following arrangement. That is, a first transmission unit that transmits an image for display on another communication device, and a display image displayed on the display unit of the communication device are transmitted for display on the other communication device. A first notification for notifying the other communication device of information relating to display of the retained image as information relating to image display when transmitting the retained image to the other communication device. And a determination to determine whether to transmit the display image to the other communication device by the second transmission unit after transmitting the retained image to the other communication device by the first transmission unit And when the holding image is transmitted to the other communication device, the second transmission unit transmits the display image to the other communication device. Above If the determination unit determines that the display image is not transmitted, the second image is transmitted to the other communication device, and then the second image is transmitted. A control unit that controls the display unit to prevent transmission of the display image to the other communication device; and the second transmission when the determination unit determines to transmit the display image. And second notification means for notifying the other communication device of information relating to display of the display image before the display image is transmitted by the means.

  According to the present invention having the above configuration, screen sharing can be performed between devices intended by a user.

FIG. 3 is a diagram showing a connection form between a transmission apparatus 100 and a reception apparatus 200 according to Embodiments 1 and 2. FIG. 3 is a block diagram showing an example of the internal configuration of the transmission apparatus 100 according to the first embodiment. FIG. 5 is a diagram showing an exemplary sequence of screen sharing processing according to the first embodiment. FIG. 5 is a diagram showing an example of a connection destination selection screen according to the first embodiment. FIG. 5 is a diagram showing an example of screen sharing confirmation determination control according to the first embodiment. FIG. 5 is a diagram showing an example of a screen sharing confirmation screen according to the first embodiment. FIG. 5 is a diagram illustrating an example of screen sharing selection determination control according to the first embodiment. FIG. 5 is a diagram showing an example of a shared screen according to the first embodiment. FIG. 5 is a block diagram showing an example of an internal configuration of a transmission apparatus 900 according to the second embodiment. FIG. 10 is a diagram showing an exemplary sequence of screen sharing processing according to the second embodiment. FIG. 9 is a diagram showing an example of a screen sharing confirmation transmission image according to the second embodiment. FIG. 10 is a diagram showing an example of a screen sharing confirmation screen display according to the second embodiment.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

<Embodiment 1>
In the present embodiment, a method for confirming transmission of a shared screen by sharing a transmission confirmation image between a transmission device and a reception device will be described.

  FIG. 1 is a diagram illustrating a connection form between the transmission device 100 and the reception device 200 according to the first embodiment. The transmission device 100 and the reception device 200 are wirelessly connected to each other via the wireless network 300. The transmission device 100 starts, pauses, or ends transmission of image data according to an image distribution request transmitted from the reception device 200 via the wireless network 300. The transmitter 100 conforms to the Wi-Fi CERTIFIED Miracast (registered trademark) standard for operations not specifically described in the present embodiment. Specific examples of the transmission device 100 include a camera, a video camera, a smartphone, a mobile phone, a PC, and the like. However, the transmission device 100 is not limited to these as long as the hardware configuration and the module configuration described below are satisfied.

  On the other hand, the receiving device 200 receives image data transmitted from the transmitting device 100 via the wireless network 300, and displays the image data. The receiving device 200 follows the Wi-Fi CERTIFIED Miracast (registered trademark) standard for operations not specifically described in the present embodiment. Specific examples of the receiving device 200 include a smartphone, a mobile phone, a PC, a television, a projector, and the like, but are not limited to these as long as the hardware configuration and the module configuration described below are satisfied. The wireless network 300 is, for example, a wireless LAN (Wireless LAN) as a home network, but is not limited thereto.

  FIG. 2 is a block diagram illustrating an internal configuration example of the transmission apparatus 100 according to the first embodiment. The bus 101 is a transfer path for various data. For example, digital data captured by the capture unit 108 is sent to a predetermined processing unit via the bus 101. The communication unit 102 transmits / receives image data and control information to / from the receiving device 200. Since image data generally has a large amount of information, the communication unit 102 uses data compressed and encoded by the encoding unit 109 when transmitting and receiving image data via a network.

  In this embodiment, RTP (Real-time Transport Protocol) is used as a data transfer protocol, and RTSP (Real Time Streaming Protocol) is used as a playback control protocol. RTP is a protocol for transmitting and receiving multimedia data such as moving images and voices in real time via a network, standardized by RFC3550 by the IETF (The Internet Engineering Task Force). UDP, HTTP, HTTPS, and TCP can be used as transport protocols in the lower layers of RTP. RTSP is a protocol for controlling streaming standardized by RFC2326, and TCP is used as a transport protocol in the lower layer of RTSP. In addition, since the encoder unit 103 of the present embodiment uses MPEG-2 TS as a multiplexing method, the communication unit 102 packetizes image data in accordance with RFC2250 that defines the RTP payload format of MPEG-2 TS data. And Note that the packetization processing by the encoder unit 103 is not limited to this.

  The display unit 103 displays video. As the display unit 103, for example, a liquid crystal display is used. The operation unit 104 receives input (operation information) from the user. As the operation unit 104, for example, a mouse, a keyboard, or a touch screen is used. The communication control unit 105 performs communication control of image data and control information transmitted to the receiving device. Further, the communication control unit 105 may have a function of receiving remote operation information transmitted from the receiving device 200 via the wireless network 300. In this case, the user instruction input by the receiving device 200 It can also be handled as an input of the unit 104.

  The display control unit 106 performs display control of images and characters displayed on the display unit 103. The operation control unit 107 controls operation information input from the operation unit 104. The capture unit 108 captures an image displayed on the display unit 103 and generates image data. The captured image includes, for example, OSD information such as operation buttons displayed so as to be superimposed on the image. The encoding unit 109 performs a process of compressing and encoding image data using the MPEG-4 AVC / H.264 format and further multiplexing the image data using the MPEG-2 TS format. MPEG-2 TS is a container format for multiplexing and transmitting media data such as encoded images and sounds, and is used as a multiplexing method of this embodiment.

  The storage unit 110 includes a ROM and a RAM that store a program for operating the transmission device 100 and control data. The application control unit 111 controls an application program that operates on the transmission device 100 stored in the storage unit 110. The confirmation image generation unit 112 generates a screen sharing confirmation screen for confirming whether the screen displayed on the transmission device 100 is shared with other devices (whether screen sharing is performed). The screen sharing confirmation unit 113 determines whether or not to perform screen sharing confirmation for allowing the user to select whether or not to perform screen sharing. The screen sharing determination unit 114 determines whether to perform screen sharing.

  Next, image transmission / reception control by the transmission device 100 and the reception device 200 of the present embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of a sequence from when the transmission device 100 and the reception device 200 start connection to when the transmission device 100 transmits an image according to the first embodiment. It is the figure which has described the step in time series order from the top to the bottom.

  In step S301, the communication control unit 105 of the transmission device 100 starts searching for a connection destination device. As a search method, for example, the transmission device 100 executes Device Discovery, which is a process defined by Wi-Fi Direct (registered trademark). In Device Discovery, a plurality of devices search for each other device by wireless frame transmission / reception using Probe Request and Probe Response which are wireless LAN management frames. The process advances to step S302 when a certain time elapses or the device search is ended by the user.

  In step S302, display unit 103 of transmission device 100 displays the list of reception devices generated by display control unit 106. The list of receiving apparatuses is a list of information that can be uniquely identified by the receiving apparatus received by the transmitting apparatus 100 from the receiving apparatus in step 301. As this information, for example, information of a P2P information element (hereinafter referred to as P2P IE) included in a Beacon frame, a Probe Response frame, or the like received from the receiving apparatus 200 is used. In addition, the P2P Device Address and Device Name information included in the P2P Device Info attribute of the P2P IE is information that can uniquely identify the receiving device.

  FIG. 4 is an example of the connection destination selection screen 400 displayed on the display unit 103 of the transmission device 100 in step S302. On the connection destination selection screen 400, a list 401 of receiving devices is displayed. Here, it is assumed that receiving apparatus 200 is Miracast_Sink-001 acquired from Device Name in receiving apparatus list 401. When an arbitrary receiving device is selected by a user operation in the receiving device list 401 and the “execute” button 404 is pressed, the connection destination selection screen 400 is closed, and the process proceeds to step S303.

  Note that the display unit 103 of the transmission device 100 may display a selection screen as to whether or not to confirm the screen sharing destination. As an example of a screen for selecting whether or not to confirm a screen sharing destination, FIG. 4 shows a screen sharing destination confirmation selection 402. The screen sharing destination confirmation selection 402 is a check box. When the screen sharing destination confirmation selection 402 is checked by the user's operation and the “execute” button 404 is pressed, the transmitting apparatus performs screen sharing confirmation described later with reference to FIG. Note that the user presses a “cancel” button 403 to cancel the selection of the receiving device in the list 401 of receiving devices and the check of the screen sharing destination confirmation selection 402.

  In this embodiment, if the receiving device 200 is selected by a user operation in step S302, the communication control unit 105 of the transmitting device 100 starts a wireless network connection with the receiving device 200 in step S303. Here, the transmitting apparatus 100 uses Wi-Fi Protected Setup (registered trademark) (hereinafter referred to as WPS) as a wireless network security setting method. Security setting methods using WPS include a PIN method using a password, a PBC method using a push button, and an NFC method using near field communication (NFC). When the wireless network connection is completed, the process proceeds to step S304.

  In step S304, communication control unit 105 of transmitting apparatus 100 exchanges capability information with receiving apparatus 200. The capability information is information on images that can be handled by the transmission device 100 or the reception device 200, information on UIBC (User Input Back Channel) devices, and the like. Here, the UIBC device information is information for the reception device 200 to remotely operate the transmission device 100. The UIBC device information includes information on whether or not the remote operation is valid, and information on the type of operation device that performs the remote operation. The operation device type is, for example, a mouse or a keyboard. The exchange of capability information is performed by, for example, RTSP message exchange in Wi-Fi CERTIFIED Miracast (registered trademark) (hereinafter referred to as Miracast). Specifically, the transmission device 100 and the reception device 200 exchange capability information according to M1 (OPTION), M2 (OPTION), M3 (GET_PARAMETER), and M4 (SET_PARAMETER) defined in Miracast.

  When exchanging capability information, the screen sharing confirmation unit 113 of the transmission device 100 may determine whether or not to transmit a screen sharing confirmation image to the reception device 200 (screen sharing confirmation determination). The screen sharing confirmation determination method will be described later with reference to FIG. In this screen sharing confirmation determination, when confirming a transmission destination device, the transmission device 100 may set reception of UIBC device information to be invalid by an RTSP M4 request message. Specifically, the transmission device 100 sets the wfd-uibc-setting parameter to disable in the RTSP M4 request. The transmission apparatus 100 can prevent the reception apparatus 200 from operating the screen sharing selection performed in a later step by setting the reception of UIBC device information to invalid. When the exchange of capability information ends, the process proceeds to step S305.

  In step S305, the communication control unit 105 of the transmission device 100 performs image transmission setup with the reception device 200. The image transmission setup is performed by, for example, RTSP message exchange in Miracast. Specifically, this message exchange is a message exchange by Trigger SETUP by M5 (SET_PARAMETER), Trigger PLAY by M5 (SET_PARAMETER), Trigger PLAY by M5 (SET_PARAMETER), and M7 (PLAY) as defined in Miracast. Here, the communication control unit 105 of the transmission device 100 branches the subsequent processing according to the result of the screen sharing confirmation determination by the screen sharing confirmation unit 113 in step S304. That is, when the screen sharing confirmation unit 113 performs screen sharing confirmation, the process proceeds to step S306. If the screen sharing confirmation is not performed, the process proceeds to step S312.

  In step S306, display unit 103 of transmitting apparatus 100 displays a screen sharing confirmation image. The screen sharing confirmation image is generated by the confirmation image generation unit 112 and displayed on the display unit 103 through the display control unit 106. The screen sharing confirmation image is, for example, an image that can be uniquely identified by the transmission device 100 as shown in FIG. The display unit 103 may display the screen sharing confirmation image on the entire screen using the entire display area. By displaying the display unit 103 using the entire screen, it is possible to prevent the display unit 103 from displaying the shared screen that the user wants to share. In addition, the shared screen is a screen displayed by an application operating in the application control unit 111 of the transmission device 100 as illustrated in FIG. After the display unit 103 displays the screen sharing confirmation image, the process proceeds to step S307.

  In step S307, the communication control unit 105 of the transmission device 100 transmits the screen sharing confirmation image to the reception device 200. Real-time Transport Protocol (hereinafter referred to as RTP) is used for image transmission. At this time, the transmission image is generated by capturing an image displayed on the transmission device 100. Specifically, first, the capture unit 108 of the transmission device 100 captures an image displayed on the display unit 103. Subsequently, the encoding unit 109 of the transmission device 100 encodes the image captured by the capture unit 108 and generates a transmission image that can be received by the reception device 200. After starting transmission of the screen sharing confirmation image, the process proceeds to step S308.

  In step S308, the screen sharing determination unit 114 of the transmission device 100 determines whether to share the screen. The determination method in the screen sharing determination unit 114 will be described later with reference to FIG. If it is determined to share the screen, the process proceeds to step S309. If it is determined not to share the screen, the process proceeds to step S311.

  In step S309, the communication control unit 105 of the transmission device 100 transmits a shared image for screen sharing to the reception device 200. The transmission and generation of the image are realized by capturing the display image by the encoding unit 109 of the transmission device 100 using RTP as in step S307. Here, the display unit 103 of the transmission device 100 displays the shared image. The shared image is an image generated by the application control unit 111, for example. FIG. 8 is a diagram illustrating an example of a shared image. Since the screen displayed on the display unit 103 of the transmission device 100 is a shared screen, the transmission image generated by the capture unit 108 is a shared image. The shared image may be an image that occupies a part of the range that can be displayed on the display unit 103. After starting transmission of the shared image, the process proceeds to step S310.

  In step S310, the communication control unit 105 of the transmission device 100 sets the UIBC device information to be valid for the reception device 200. Specifically, the communication control unit 105 sets the wfd-uibc-setting parameter to enable in the RTSP M4 request. The communication control unit 105 permits the remote operation from the receiving device 200 thereafter by setting the UIBC device information to be valid.

  If it is determined in step 308 that screen sharing is not to be performed, in step S311, the communication control unit 105 of the transmission device 100 disconnects the wireless network connection with the reception device 200 and ends this sequence. If it is determined in step S304 that the screen sharing confirmation is not performed, the communication control unit 105 of the transmission device 100 transmits the shared image to the reception device 200 in step S312. Since the method for generating and transmitting the shared image is the same as that in step S309, description thereof is omitted here.

  FIG. 5 is a diagram illustrating an example of screen sharing confirmation determination control of the transmission device 100 in step S304. The screen sharing confirmation determination control is performed by the screen sharing confirmation unit 113 in FIG. 2, and it is determined whether or not to perform screen sharing confirmation. In step S501, the screen sharing confirmation unit 113 of the transmission device 100 determines whether or not a screen sharing confirmation condition is satisfied. If the screen sharing confirmation condition is satisfied, the screen sharing confirmation unit 113 determines to perform screen sharing confirmation (step S502). If the screen sharing confirmation condition is not satisfied, the screen sharing confirmation unit 113 determines not to perform the screen sharing confirmation (step S503). Here, the screen sharing confirmation condition is, for example, any one or combination of the following four conditions.

  The first is that the screen sharing destination confirmation selection 402 on the connection destination selection screen 400 in FIG. 4 is checked (when “Confirm screen sharing destination” is selected). Second, the WPS security setting method used when the communication unit 102 of the transmission device 100 establishes a wireless network connection is a specific method. Specifically, this is a case where the security setting method is a PBC method in which push buttons are used. When the security setting method uses the PIN method for displaying the password on the receiving device or the NFC method for specifying the connection destination by a touch operation, the screen sharing confirmation unit 113 determines that the screen sharing confirmation is not performed. Third, the image data type displayed by the application control unit 111 of the transmission apparatus 100 is a confidential data type. Fourth, the image data type displayed by the application control unit 111 of the transmission device 100 is a data type handled as copyright protection.

  FIG. 7 is a diagram illustrating an example of screen sharing determination control of the transmission device 100 in step S308. The screen sharing determination control is performed by the screen sharing determination unit 114 in FIG. FIG. 6 shows an example of the screen sharing confirmation screen. The operation control unit 107 of the transmission device 100 detects a button press by the user on the screen sharing confirmation screen displayed on the display unit 103. Based on this detection, the screen sharing determination unit 114 determines whether to perform screen sharing. Determine whether. The user visually recognizes the screen sharing image displayed on the receiving apparatus 200, determines whether to permit screen sharing, and presses the button.

  In step S701, the operation control unit 107 determines that the “Yes” button 601 has been pressed on the screen sharing confirmation screen. If the “Yes” button 601 is pressed, the screen sharing determination unit 114 determines to perform screen sharing (step S702). If the “No” button 602 has been pressed, the screen sharing determination unit 114 determines that screen sharing is not performed, and performs processing for disconnecting the wireless network connection (step S703). If the communication control unit 105 of the transmission device 100 can detect the user instruction input by the reception device 200, the screen sharing determination unit 114 determines whether to perform screen sharing in response to the detection. May be.

  As described above, according to the present embodiment, the transmission device transmits a screen sharing confirmation image for visually recognizing the image transmission destination to the user before screen sharing. The user determines whether to allow screen sharing by visually recognizing the screen sharing confirmation image displayed on the receiving device that performs screen sharing. Accordingly, it is possible to reduce the possibility of transmitting the shared image to a receiving device that is not desired by the user.

<Embodiment 2>
In the present embodiment, a method for confirming transmission of a shared screen by transmitting a transmission confirmation image created in advance by the transmission apparatus will be described. Regarding points similar to those of the first embodiment, description thereof is omitted here.

  Since FIG. 1 is the same as that of the first embodiment, a description thereof will be omitted. FIG. 9 is a block diagram illustrating an internal configuration example of the transmission apparatus 900 according to the second embodiment. As shown in FIG. 9, the transmission device 900 of this embodiment includes a confirmation image storage unit 901. In the figure, the same parts as those in FIG.

  The confirmation screen storage unit 901 holds a transmission confirmation image to be transmitted to the receiving device 200. The screen sharing confirmation image is image data that has been compression-encoded by the MPEG-4 AVC / H.264 system and further multiplexed by the MPEG-2 TS system. The screen sharing confirmation image is an image that can be uniquely identified by the transmission device 900, such as a screen sharing confirmation transmission image 1100 shown in FIG. Further, regarding the generation of the screen sharing confirmation image, the transmission device 900 may hold an image that has been generated in advance in the ROM, or the encoding may be performed when the transmission device 900 starts up the system or when the sequence illustrated in FIG. The image dynamically generated by the unit 109 may be held in the RAM. Since the screen sharing confirmation image format must be able to be displayed on all receiving devices, it is desirable to use an image format that is required to be supported by Miracast.

  Next, image transmission / reception control by the transmission device 900 and the reception device 200 of the present embodiment will be described with reference to FIG. FIG. 10 is a diagram illustrating an example of a sequence from when the transmission device 900 and the reception device 200 start connection to when the transmission device 900 transmits an image according to the second embodiment. It is the figure which has described the step in time series order from the top to the bottom.

  In step S1001, communication control unit 105 of transmitting apparatus 900 starts searching for a connection destination apparatus. The processing in this step is the same as that in step S301 in FIG. The process advances to step S1002 when a certain time elapses or the device search end operation is performed by the user. In step S1002, display unit 103 of transmitting apparatus 900 displays the list of receiving apparatuses generated by display control unit 106. The processing in this step is the same as that in step S302 in FIG. When a receiving device is selected by a user operation in the receiving device list 401, the connection destination selection screen 400 is closed, and the process proceeds to step S1003.

  In step S1003, communication control section 105 of transmitting apparatus 900 starts a wireless network connection with receiving apparatus 200. The processing in this step is the same as that in step S303 in FIG. When the wireless network connection is completed, the process proceeds to step S31004. In step S1004, communication control unit 105 of transmitting apparatus 900 exchanges capability information with receiving apparatus 200. This step is the same as the capability information exchange method in step S304 of FIG. As the transmission image information to be transmitted with the wfd-video-formats parameter of M4 (SET_PARAMETER), the image resolution and refresh rate information of the shared confirmation image held in the confirmation image storage unit 901 to be transmitted in the subsequent step S1007 are used. The point is different. When the exchange of capability information ends, the process proceeds to step S1005.

  In step S1005, communication control unit 105 of transmitting apparatus 900 performs image transmission setup with receiving apparatus 200. The processing in this step is the same as that in step S305 in FIG. If screen sharing confirmation is to be performed, the process proceeds to step S1006. If the screen sharing confirmation is not performed, the process proceeds to step S1013. In step S1006, display unit 103 of transmitting apparatus 900 displays a screen sharing confirmation image. The screen sharing confirmation image is generated by the confirmation image generation unit 112 of the transmission device 900 and displayed on the display unit 103 through the display control unit 106. Here, as in the first embodiment, the screen sharing confirmation image is an image that can be uniquely identified by the transmission device 900 as shown in FIG.

  FIG. 12 shows a display example of the screen sharing confirmation screen on the display unit 103. In the screen sharing confirmation screen display 1200 of FIG. 12, the screen sharing confirmation image 600 is displayed on the lower right side where the shared image 800 is easily visible. As a result, the user can simultaneously confirm the screen for screen sharing and the screen sharing confirmation screen. After the display unit 103 displays the screen sharing confirmation image, the process proceeds to step S1007.

  In step S1007, the communication control unit 105 of the transmission device 900 transmits the transmission confirmation image stored in the confirmation image storage unit 901 to the reception device 200. FIG. 11 shows an example of the transmission confirmation image 1100. As shown in FIG. 11, the transmission confirmation image 1100 is an image that can be uniquely identified by the transmission apparatus 900. The image transmission uses RTP as in step S307 in FIG. After starting transmission of the transmission confirmation image 1100, the transmission apparatus 100 proceeds to step S1008.

  In step S1008, the screen sharing determination unit 114 of the transmission device 100 determines whether to share the screen. The processing in this step is the same as that in step S308 in FIG. If it is determined to share the screen, the process proceeds to step S1009. If it is determined not to share the screen, the process proceeds to step S1012.

  In step S1009, the communication control unit 105 of the transmission device 900 notifies the reception device 200 of the change of the AV format in order to switch the transmission image. The change of AV format is notified by RTSP M4 (SET_PARAMETER) message. Specifically, the RTSP M4 request uses the wfd-video-formats parameter to notify image information such as the resolution and refresh rate of the shared image to be transmitted in the subsequent step S1010. When the notification of the AV format ends, the process proceeds to step S1010.

  In step S1010, communication control unit 105 of transmitting apparatus 900 transmits a shared image for screen sharing to receiving apparatus 200. The processing in this step is the same as that in step S309 in FIG. After transmission of the shared image starts, the process proceeds to step S1011. In step S1011, communication control section 105 of transmitting apparatus 900 sets UIBC device information to receiving apparatus 200 as valid. The processing in this step is the same as that in step S310 in FIG. In step S1012, the communication control unit 105 of the transmission device 900 disconnects the wireless network connection with the reception device 200, and ends this sequence. In step S1013, the communication control unit 105 of the transmission device 900 transmits the shared image to the reception device 200. Since the method for generating and transmitting the shared image is the same as that in step S309, description thereof is omitted here.

  As described above, according to the present embodiment, the transmission apparatus transmits a transmission confirmation screen prepared in advance before screen sharing. A screen sharing confirmation image displayed on the receiving apparatus that performs screen sharing is visually recognized to determine whether or not to permit screen sharing. Accordingly, it is possible to reduce the possibility of transmitting the shared image to a receiving device that is not desired by the user. Accordingly, it is possible to reduce the possibility of transmitting the shared image to a receiving device that is not desired by the user.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 transmitter, 101 bus, 102 communication unit, 103 display unit, 104 operation unit, 105 communication control unit, 106 display control unit, 107 operation control unit, 108 capture unit, 109 encoding unit, 110 storage unit, 111 application control unit 112 Confirmation image generation unit 113 Screen sharing confirmation unit 114 Screen sharing determination unit 200 Receiver device 300 Wireless network

Claims (12)

A communication device,
First transmission means for transmitting a retained image that is stored in the storage unit of the communication device and is not displayed on the display unit of the communication device in order to display it on another communication device;
A second transmission means for transmitting a display image displayed on the display unit of the communication device for displaying on the other communication device;
First notification means for notifying the other communication device of information relating to the display of the retained image as information relating to the display of the image when transmitting the retained image to the other communication device;
Determining means for determining whether to transmit the display image to the other communication device by the second transmission means after transmitting the retained image to the other communication device by the first transmission means;
When it is determined by the determining means that the display image is to be transmitted, the display image to the other communication apparatus by the second transmitting means after the retained image is transmitted to the other communication apparatus. Control to be sent,
If it is determined by the determining means that the display image is not to be transmitted, the display on the other communication apparatus by the second transmitting means is performed after the retained image is transmitted to the other communication apparatus. Control means for controlling the transmission of images not to be performed;
If it is determined by the determining means that the display image is to be transmitted, before the display image is transmitted by the second transmitting means, information related to the display image display is sent to the other communication device. A second notification means for notification;
A communication apparatus comprising:   The communication apparatus according to claim 1, wherein the information regarding the display of the retained image notified by the first notification unit is at least one of a resolution and a refresh rate of the retained image.   The communication apparatus according to claim 1, wherein the information related to display of the display image notified by the second notification unit is at least one of a resolution and a refresh rate of the display image. When transmitting the retained image by the first transmitting unit, an image that is different from the retained image and that accepts a user operation as to whether or not to transmit the display image by the second transmitting unit. It further has a display control means for displaying on the display unit,
4. The determination unit according to claim 1, wherein the determination unit determines whether to transmit the display image by the second transmission unit based on the received user operation. 5. Communication equipment.   The communication apparatus according to claim 1, wherein the retained image is an image that can uniquely identify the communication apparatus. Further comprising setting means for invalidating reception of information for remote operation from the other communication device;
The first transmission unit transmits the retained image after the setting unit has disabled the reception of information for the remote operation from the other communication device. The communication device according to claim 5. The image processing apparatus further includes acquisition means for capturing image data displayed on the display unit and acquiring image data
The communication apparatus according to any one of claims 1 to 6, wherein the second transmission unit transmits the image data acquired by the acquisition unit.   The communication device transmits the retained image by the first transmission unit via a wireless connection with the other communication device, and the determination unit transmits the display image by the second transmission unit. The communication apparatus according to any one of claims 1 to 7, wherein the wireless connection is disconnected when it is determined that the wireless connection is not performed.   The communication apparatus according to any one of claims 1 to 8, wherein the first transmission unit transmits the retained image in accordance with a Wi-Fi Miracast standard.   The communication apparatus according to claim 1, wherein the second transmission unit transmits the display image in accordance with a Wi-Fi Miracast standard. A communication device control method comprising:
A first transmission step of transmitting a retained image that is stored in the storage unit of the communication device and is not displayed on the display unit of the communication device in order to be displayed on another communication device;
A second transmission step of transmitting the display image displayed on the display unit of the communication device to display on the other communication device;
A first notification step of notifying the other communication device of information relating to display of the retained image as information relating to image display when transmitting the retained image to the other communication device;
A determination step of determining whether to transmit the display image to the other communication device in the second transmission step after transmitting the retained image to the other communication device in the first transmission step;
When it is determined that the display image is transmitted in the determining step, the display image to the other communication device in the second transmission step is transmitted after the retained image is transmitted to the other communication device. Control to be sent,
If it is determined not to transmit the display image in the determination step, the display to the other communication device in the second transmission step is performed after the retained image is transmitted to the other communication device. A control process for controlling the transmission of images not to be performed;
If it is determined in the determining step that the display image is to be transmitted, before the display image is transmitted in the second transmitting step, information related to the display image display is transmitted to the other communication device. A second notification step to notify;
A communication apparatus comprising:   The program for functioning a computer as each means of the communication apparatus as described in any one of Claim 1 to 10.

Images