JP6604256B2 - Information processing terminal, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing terminal, an information processing method, and a program.

  Smart terminals such as smartphones and tablets have become widespread and are used in business. A smart terminal usually includes a camera, a microphone, a speaker, and the like. In view of this, it is conceivable to share an image of a work site taken by a smart terminal with a PC (Personal Computer) in a remote office or the like. By doing so, a skilled person located in an office or the like can remotely support the work of the worker at each of a plurality of work sites. As a result, the knowledge and experience of a few skilled workers can be efficiently applied to a plurality of work sites.

  Conventionally, a video conference system is known as a system for sharing images and sounds between remote locations. By using such a video conference system mechanism for smart terminals used by workers and PCs (Personal Computers) used by skilled workers, it is possible to easily construct a system for remote support by skilled workers. It is believed that there is.

JP 2006-140747 A JP 2014-045438 A JP 09-00922 A

  However, in general, in a video conference system, it is assumed that terminals serving as communication partners are of the same model and are used in a similar network environment. Therefore, such a video conference system does not sufficiently consider collaboration between a smart terminal used by a worker at a work site and a PC used by a skilled worker in an office or the like.

  Specifically, smart terminals at work sites often use wireless communications such as 3G, LTE (Long Term Evolution), or Wi-Fi (registered trademark). Wireless communication has a large variation in available bandwidth depending on radio wave conditions and time zones, and has a larger delay than wired communication. As a result, there is a possibility that a lot of sound breaks out and that the other party is hard to hear the sound.

  In particular, in a situation in which dialogue is performed by voice while sending an image, if the voice quality is poor, it becomes difficult for a skilled person to grasp the situation in the field, and work efficiency deteriorates.

  An object of this invention is to suppress the fall of the quality of the audio | voice transmitted with an image.

  In one aspect, the information processing terminal includes an image input unit that inputs image data, an audio input unit that inputs audio data, an encoding unit that encodes the image data, and the encoded image data A transmission unit that transmits the audio data to an information processing apparatus connected via a network; and an estimation unit that acquires an estimated value of a bandwidth that can be used for transmitting the image data and the audio data in the network. Then, the encoding unit encodes the image data so as to be within a band smaller than the estimated value.

  As one aspect, it is possible to suppress a decrease in quality of sound transmitted together with an image.

It is a figure which shows the structural example of the work assistance system in embodiment of this invention. It is a figure which shows the hardware structural example of the worker terminal in embodiment of this invention. It is a figure which shows the function structural example of each apparatus in embodiment of this invention. It is a figure for demonstrating the movement state of an operator. It is a figure for demonstrating the transfer control method of the image data in this Embodiment. It is a flowchart for demonstrating an example of the process sequence which an operator terminal performs. It is a flowchart for demonstrating an example of the process sequence which a supporter terminal performs. It is a figure for demonstrating an example of the estimation process of an available band.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a configuration example of a work support system according to an embodiment of the present invention. In FIG. 1, the work support system 1 includes a worker terminal 10, a supporter terminal 20, and the like. The worker terminal 10 and the supporter terminal 20 are communicably connected via the IP network N1 or the like.

  The worker terminal 10 is a terminal possessed and used by the worker. For example, a smart terminal such as a smartphone or a tablet terminal may be used as the worker terminal 10. Alternatively, a glasses-type terminal, a wearable terminal that is fixed to the worker's head and can photograph the direction of the operator's line of sight, and the like may be used as the worker terminal 10.

  An operator means a person who performs work at a work site. The worker terminal 10 inputs an image (moving image) related to a work target or the like, a worker's voice, and the like, and transmits the image and sound to the supporter terminal 20 via the network N1. The worker terminal 10 also outputs a voice transmitted from the supporter terminal 20. The worker terminal 10 is connected to the IP network N1 via a wireless communication network N2 such as 3G, LTE (Long Term Evolution), or Wi-Fi (registered trademark).

  The supporter terminal 20 is a terminal used by the supporter. The supporter refers to a person who remotely supports the work performed by the worker while viewing an image or sound transmitted from the worker terminal 10. The supporter terminal 20 is, for example, a PC (Personal Computer), a tablet terminal, or a smartphone.

  Note that the work support system 1 may include a plurality of worker terminals 10 and a plurality of supporter terminals 20.

  FIG. 2 is a diagram illustrating a hardware configuration example of the worker terminal according to the embodiment of the present invention. The worker terminal 10 in FIG. 2 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, a display device 106, an input device 107, a camera 108, and a microphone, which are mutually connected by a bus B. 109, a speaker 110, and an acceleration sensor 111.

  A program that realizes processing in the worker terminal 10 is provided by the recording medium 101. When the recording medium 101 on which the program is recorded is set in the drive device 100, the program is installed from the recording medium 101 to the auxiliary storage device 102 via the drive device 100. However, the program need not be installed from the recording medium 101 and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 realizes functions related to the worker terminal 10 in accordance with a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network. The display device 106 displays a GUI (Graphical User Interface) or the like by a program. The input device 107 is, for example, a touch panel or a button, and is used for inputting various operation instructions.

  The camera 108 is a digital video camera, for example, and captures (inputs) an image of a work object or the like. The microphone 109 inputs a voice such as a worker's voice. The speaker 110 outputs sound based on the sound signal transmitted from the supporter terminal 20. The acceleration sensor 111 detects vibration that occurs in response to the movement of the worker terminal 10 or the like.

  The supporter terminal 20 may also have a hardware configuration as shown in FIG. However, the supporter terminal 20 may not include the camera 108 and the acceleration sensor 111.

  FIG. 3 is a diagram illustrating a functional configuration example of each device according to the embodiment of the present invention. In FIG. 3, the worker terminal 10 includes an input / output control unit 121, an audio compression unit 122, an audio transmission / reception unit 123, an audio reproduction unit 124, an image display unit 125, a display image acquisition unit 126, an image difference detection unit 127, an image compression. 128, image transmission control unit 129, wireless communication control unit 130, delay detection unit 131, band estimation unit 132, image quality adjustment unit 133, movement state determination unit 134, and the like. Each of these units is realized, for example, by processing that the CPU 104 causes one or more programs installed in the worker terminal 10 to execute.

  The input / output control unit 121 performs control of input of image data (video data) from the camera 108, control of input of sound data from the microphone 109, control of output of sound from the speaker 110 based on the sound data, and the like. .

  The audio compression unit 122 compresses (encodes) audio data input from the microphone 109 by the input / output control unit 121. The voice transmission / reception unit 123 performs transmission control on the voice data encoded by the voice compression unit 122 to the supporter terminal 20 or performs reception control on the voice data received from the supporter terminal 20. The audio reproduction unit 124 reproduces (for example, decodes) the audio data received by the audio transmission / reception unit 123.

  The image display unit 125 displays image data captured by the camera 108 and input by the input / output control unit 121 on the display device 106.

  The display image acquisition unit 126 acquires image data displayed on the display device 106 by screen capture.

  The image difference detection unit 127 includes a range of differences between the image data acquired by the display image acquisition unit 126 and the image data that has already been transmitted to the supporter terminal 20 (for example, at the time of the previous transmission) (hereinafter, “difference image”). ) Is detected. That is, a technique for transferring only the difference image used in the known screen sharing technique is also used in the present embodiment. The use of this technology is not essential.

  The image compression unit 128 encodes (compresses) the difference image. The image transmission control unit 129 controls transmission of the encoded difference image to the supporter terminal 20.

  The wireless communication control unit 130 controls wireless communication regarding audio data and image data.

  The delay detection unit 131 detects the occurrence of a communication delay between the worker terminal 10 and the supporter terminal 20. In the present embodiment, the occurrence of the delay is detected by referring to the audio data reception buffer included in the audio transmission / reception unit 123. For example, when the reception buffer becomes empty in a state where all the audio data scheduled to be received has not been received, the delay detection unit 131 detects the occurrence of a delay. The data size of the audio data scheduled to be received can be specified by referring to the header of the audio data, for example. However, the communication delay may be detected by other methods.

  The band estimation unit 132 estimates a band that can actually be used in communication with the supporter terminal 20.

  The image quality adjustment unit 133 adjusts the image quality of the difference image transmitted to the supporter terminal 20. In the present embodiment, the “image quality” does not include the frame rate (FPS). Specifically, “image quality” refers to the image quality for each frame, and is an attribute that varies depending on encoding.

  The movement state determination unit 134 determines the movement state of the worker based on the vibration of the worker terminal 10 detected by the acceleration sensor 111. For example, the movement vector and the movement speed are calculated as parameters for determining the movement state. The movement state determination unit 134 determines whether or not the operator is photographing the work target based on whether or not these parameters satisfy a predetermined condition. Details of the movement state will be described later. The worker terminal 10 moves with the worker. Therefore, the movement state of the worker is equivalent to the movement state of the worker terminal 10.

  On the other hand, the supporter terminal 20 includes an input / output control unit 21, a communication control unit 22, a voice transmission / reception unit 23, a differential image reception unit 24, an image display unit 25, and the like. Each of these units is realized by processing that one or more programs installed in the supporter terminal 20 cause the CPU of the supporter terminal 20 to execute.

  The input / output control unit 21 controls input of voice data from a microphone connected to the supporter terminal 20, output of voice data to a speaker connected to the supporter terminal 20, and the like.

  The communication control unit 22 controls communication with the worker terminal 10.

  The voice transmission / reception unit 23 controls transmission of voice data input by the input / output control unit 21 to the worker terminal 10, reception of voice data transmitted from the worker terminal 10, and the like.

  The difference image receiving unit 24 controls reception of image data (difference image) transmitted from the worker terminal 10. The image display unit 25 controls display of image data transmitted from the worker terminal 10.

  Next, the movement state of the worker terminal 10 will be described. FIG. 4 is a diagram for explaining the movement state of the worker. In FIG. 4, the movement state of the worker is classified into a state A, a state B, and a state C.

  State A is a state in which the operator is moving to the equipment to be inspected. In this case, the vibration detected by the acceleration sensor 111 of the worker terminal 10 is small due to hand shake or the like, and the average movement vector tends to be small.

  State B is a state in which the worker has arrived at the facility to be inspected and is looking for an inspection location. In this case, the acceleration sensor 111 detects vibration according to a change in the visual line direction of the worker. Therefore, the average movement vector tends to be a medium value.

  The state B is a state that appears prominently when the worker terminal 10 is a wearable terminal.

  State C is a state in which the operator takes an image of the inspection location and asks for assistance from the supporter. Since the range that can be photographed by the camera 108 of the worker terminal 10 is small, the worker needs to move the worker terminal 10 in parallel in order to photograph the entire inspection location that is wide with respect to the photographable range. Yes. Therefore, in the state C, the average movement vector tends to increase.

As is apparent from the above, the dialogue between the worker and the supporter is mainly performed in the state C. Therefore, in the present embodiment, in the state C, in order to facilitate communication between the worker and the supporter, control is performed according to the following policy.
(1) When a communication delay is detected, the transfer amount of image data is reduced, and a bandwidth of audio data is secured as much as possible.
(2) A reduction in the transfer amount of image data is realized not by a reduction in frame rate but by a reduction in image quality.
(3) However, in the case other than the state C, the transfer amount of the image data is reduced by the reduction of the frame rate.

  In the above, (1) is based on the empirical fact that sound is more important than image (video) in communication between the worker and the supporter. That is, if the voice is interrupted, the intention of the other party cannot be grasped and the conversation becomes difficult, but the possibility that the conversation becomes difficult due to the deterioration of the image quality is low. In other words, it is possible to compensate for the deterioration in image quality through dialogue. Further, if the image quality is temporarily lowered, the influence on communication is small.

  (2) is based on an empirical fact that an unnatural motion image has a greater negative effect on support than a blurred image. In other words, if the frame rate is lowered in order to reduce the transfer amount of image data, the reproduced video becomes unnatural. On the other hand, when the image quality is lowered, the reproduced video is blurred, but the movement of the image is smooth.

  In (3), the communication with the supporter is less important except in the state C, and reducing the frame rate reduces the transfer amount of the image data and reduces the battery consumption of the worker terminal 10. This is because the reduction effect is great.

  Therefore, in this embodiment, the transfer of image data is controlled as shown in FIG. FIG. 5 is a diagram for describing a method for controlling transfer of image data according to the present embodiment. In FIG. 5, a curve L1 indicates a band that can actually be used. A curve L2 represents the transfer amount of an image (video) and sound. A curve L3 indicates the transfer amount of only the voice among the transfer amounts indicated by the curve L2. In FIG. 5, the ratio between the image transfer amount and the sound transfer amount is not necessarily accurate.

  As shown in FIG. 5, in the present embodiment, when the available bandwidth indicated by the curve L1 decreases and a delay occurs in communication, the transfer amount indicated by the curve L2 decreases, but the transfer amount indicated by the curve L1. Is suppressed from decreasing as indicated by the broken line. This means that the transfer amount of images is reduced so as to be within the usable bandwidth. That is, as the image transfer amount decreases, the decrease in the audio transfer amount can be suppressed as indicated by the arrow a1. Note that the reduction in image transfer amount is realized by the reduction in image quality.

  Hereinafter, processing procedures executed by the worker terminal 10 and the supporter terminal 20 will be described. FIG. 6 is a flowchart for explaining an example of a processing procedure executed by the worker terminal.

  In step S <b> 101, the voice transmission / reception unit 123 and the image transmission control unit 129 connect communication with the supporter terminal 20. Subsequently, the image transmission control unit 129 sets the initial value of the frame rate (FPS) of the image data to 30 (S102). The value of 30 is an example. The initial value is stored in the auxiliary storage device 102, for example.

  Subsequently, the input / output control unit 121 starts photographing using the camera 108 (S103). As a result, image data captured by the camera 108 is input by the input / output control unit 121. The image data is displayed on the display device 106 by the image display unit 125.

  Subsequently, the display image acquisition unit 126 executes screen capture for the display device 106 to acquire image data displayed on the display device 106 (S104). Subsequently, the image difference detection unit 127 detects a difference image between the image data and the last transmitted image data. When the image data has not been transmitted, the entire range of the image data acquired in step S104 is detected as a difference image. The detection of the difference image can be performed using a known technique.

  Subsequently, if there is no voice input (No in S106), the image compression unit 128 encodes (compresses) the difference image (S107). At this time, the image compression unit 128 encodes the difference image so as to be equal to or less than the band (bit rate) designated by the image quality adjustment unit 133. For example, by encoding in a CBR (Constant Bitrate) mode among the encoding modes of a moving picture codec such as MPEG / H264, a differential image can be encoded so as to have a designated band. Initially, a default band (bit rate) set in advance as an initial value is designated.

  Subsequently, the image transmission control unit 129 transmits the encoded difference image to the supporter terminal 20 (S108). Subsequently, the bandwidth estimation unit 132 performs an available bandwidth estimation process based on the time required for transmission of the difference image (S109). In the usable band estimation process, an estimated value of the usable band is acquired. An example of the available bandwidth estimation process will be described later.

  Subsequently, the image transmission control unit 129 changes the frame rate based on the estimated value of the available bandwidth (S110). For example, the frame rate is set so that the transmission amount of the image data is within the band obtained by subtracting a predetermined value (for example, a high band (eg, 128 kbps) necessary for transmission of audio data) from the estimated value of the available band. Is changed.

  When the voice data from the microphone 109 is input by the input / output control unit 121, such as when the worker speaks to the supporter in the process of step S104 and subsequent steps being repeated (Yes in S106), Step S111 and subsequent steps are executed.

  In step S111, the movement state determination unit 134 determines whether vibration is detected by the acceleration sensor 111 (S111). When vibration is not detected (No in S111), Step S107 and subsequent steps are continued. When vibration is detected (Yes in S111), the movement state determination unit 134 calculates the movement vector (average movement vector) and movement speed of the worker terminal 10 based on the vibration detection value by the acceleration sensor 111. (S112). Subsequently, the movement state determination unit 134 determines whether or not the calculated movement vector is greater than or equal to a threshold value α and the calculated movement speed is greater than or equal to a threshold value β (for example, 10 cm / second) (S113). ). This determination corresponds to determination of whether or not the current movement state corresponds to the state C.

  When the calculated movement vector is less than the threshold value α, or when the calculated movement speed is less than the threshold value β (for example, 10 cm / second), that is, when the movement state does not correspond to the state C (No in S113). Step S107 and subsequent steps are continued. On the other hand, when the calculated movement vector is greater than or equal to the threshold value α and the calculated movement speed is greater than or equal to the threshold value β, that is, when the movement state corresponds to the state C (Yes in S113), the worker terminal 10 With respect to the communication with the supporter terminal 20, it is determined whether or not the delay detection unit 131 detects the occurrence of a delay of a predetermined time (for example, 50 milliseconds) or more (S114).

  When it is detected that a delay of a predetermined time or more has occurred (Yes in S114), the image quality adjustment unit 133 compares the difference with the estimated value of the available bandwidth calculated in step S109 executed last. The image quality is adjusted (S115). For example, the image quality adjustment unit 133 performs image compression on a band (bit rate) obtained by subtracting a predetermined value (for example, a high band (eg, 128 kbps) necessary for audio data transmission) from the estimated value of the available band. Part 128. Therefore, if the available band decreases, the band specified for the image compression unit 128 decreases, and if the available band increases, the band specified for the image compression unit 128 increases.

  On the other hand, if it is not detected that a delay of a predetermined time or more has occurred (No in S114), the image quality adjustment unit 133 uses the default band (bit rate) set as an initial value in advance as the image compression unit 128. (S116).

  In step S107 after the execution of step S115 or S116, the image compression unit 128 compresses (encodes) the difference image so that it finally becomes equal to or less than the band designated by the image quality adjustment unit 133.

  In the process in which step S104 and subsequent steps are repeated, for example, when an instruction to end shooting is input by the operator (Yes in S117), the processing in FIG. 6 ends.

  FIG. 7 is a flowchart for explaining an example of a processing procedure executed by the supporter terminal.

  In step S <b> 201, the voice transmission / reception unit 23 and the difference image reception unit 24 connect communication with the worker terminal 10. Thereafter, when an event occurs, processing corresponding to the event is executed.

  For example, when the difference image is received by the difference image receiving unit 24 (Yes in S202), the image display unit 25 reflects the difference image on the current display image (S203). As a result, the supporter can see the image taken by the worker terminal 10.

  When the audio transmission / reception unit 23 receives audio data (Yes in S204), the input / output control unit 21 reproduces the audio data (S205).

  When the input / output control unit 21 inputs voice data (supporter's voice) (Yes in S206), the voice transmitting / receiving unit 23 transmits the voice data to the worker terminal 10 (S207). As a result, the supporter can give instructions to the worker.

  Next, details of step S109 in FIG. 6 will be described. FIG. 8 is a diagram for explaining an example of the estimation process of the available bandwidth. The content of FIG. 8 applies the outline | summary of the content disclosed in Unexamined-Japanese-Patent No. 2013-238993 to this Embodiment.

  First, the worker terminal 10 transmits the transmission time of the worker terminal 10 to the supporter terminal 20 at the start of communication (S31). The supporter terminal 20 calculates a time difference Δt between the transmission time and the reception time (S32).

  For example, before and after step S31, the communication delay between the worker terminal 10 and the supporter terminal 20 is measured. The communication delay can be measured using, for example, a ping command. For example, by subtracting the communication delay from the time difference Δt, a difference between the time at the worker terminal 10 and the time at the supporter terminal 20 (hereinafter referred to as “time difference”) is calculated.

  Subsequently, the worker terminal 10 adds a transmission time when transmitting the image data (S33). When receiving the image data, the supporter terminal 20 calculates a transfer rate based on the time difference between the added transmission time and the reception time and the data size of the received image data (S34). At this time, the time difference is corrected by the time difference.

  Similarly in step S35, the worker terminal 10 adds a transmission time when transmitting image data. When receiving the image data, the supporter terminal 20 calculates an expected reception time of the image data based on the transfer speed calculated in step S34 and the data size of the received image data (S36). Subsequently, the supporter terminal 20 determines whether or not the expected reception time is later than the actual reception time by exceeding the threshold (S37).

  When the expected reception time is later than the actual reception time by more than the threshold, the supporter terminal 20 estimates the transfer rate obtained based on the transmission time, the actual reception time, and the data size in step S35. The value is notified to the worker terminal 10 as a value (S38). The band estimation unit 132 of the worker terminal 10 receives the estimated value.

  As described above, according to the present embodiment, the available bandwidth is estimated, and the image data is encoded so as to be within the bandwidth obtained by subtracting the bandwidth necessary for audio data transmission from the available bandwidth. Therefore, it is possible to suppress a decrease in quality of sound transmitted together with the image.

  As a result, in the work support system 1 that transmits and receives images and sound between the worker terminal 10 and the supporter terminal 20, the image captured by the worker terminal 10 is shared in real time between the site and the office, The supporter can accurately hear the explanation given by the worker.

  That is, since the quality of voice communication between a worker on site and a supporter such as an office can be maintained and sound cuts due to insufficient processing capability of the worker terminal 10 can be suppressed, smooth communication can be maintained.

  In particular, in scenes where workers are performing inspection work while photographing work objects on site, while maintaining sound quality and frame rate, the image quality is temporarily reduced, and a smooth image with a high frame rate, By transmitting high-quality sound to the supporter, remote support by the supporter can be performed smoothly.

  In the present embodiment, worker terminal 10 is an example of an information processing terminal. The supporter terminal 20 is an example of an information processing device. The input / output control unit 121 is an example of an image input unit and an audio input unit. The image compression unit 128 is an example of an encoding unit. The image transmission control unit 129 and the audio transmission / reception unit 123 are examples of a transmission unit. The band estimation unit 132 is an example of an estimation unit. The delay detection unit 131 is an example of a first detection unit. The movement state determination unit 134 is an example of a determination unit. The acceleration sensor 111 is an example of a second detection unit.

  As mentioned above, although the Example of this invention was explained in full detail, this invention is not limited to such specific embodiment, In the range of the summary of this invention described in the claim, various deformation | transformation・ Change is possible.

Regarding the above description, the following items are further disclosed.
(Appendix 1)
An image input unit for inputting image data;
A voice input unit for inputting voice data;
An encoding unit for encoding the image data;
A transmission unit for transmitting the encoded image data and the audio data to an information processing apparatus connected via a network;
An estimation unit that acquires an estimated value of a bandwidth that can be used for transmission of the image data and the audio data in the network;
The encoding unit encodes the image data so as to be within a band smaller than the estimated value;
An information processing terminal characterized by that.
(Appendix 2)
A first detection unit that detects a delay in communication via the network;
When the delay is detected, the encoding unit encodes the image data so as to be within a band smaller than the estimated value.
The information processing terminal according to Supplementary Note 1, wherein
(Appendix 3)
A second detection unit for detecting vibration of the information processing terminal;
A determination unit that determines a movement state of the information processing terminal based on the detected vibration;
The encoding unit encodes the image data so that it falls within a band smaller than the estimated value when the delay is detected and the movement state determined by the determination unit satisfies a predetermined condition. ,
The information processing terminal according to supplementary note 2, characterized by:
(Appendix 4)
Processing to input image data;
Processing to input audio data;
A process of encoding the image data; a process of transmitting the encoded image data and the audio data to an information processing apparatus connected via a network;
The information processing terminal executes processing for obtaining an estimated value of a bandwidth that can be used for transmission of the image data and the audio data in the network,
The encoding process encodes the image data so as to be within a band smaller than the estimated value.
An information processing method characterized by the above.
(Appendix 5)
The information processing terminal executes a process of detecting a delay in communication via the network,
The encoding process encodes the image data so that it falls within a band smaller than the estimated value when the delay is detected.
The information processing method according to appendix 4, characterized in that:
(Appendix 6)
Processing for detecting vibration of the information processing terminal;
The information processing terminal executes a process of determining a movement state of the information processing terminal based on the detected vibration,
The encoding process is performed when the delay is detected, and when the moving state determined in the determination process satisfies a predetermined condition, the image data is stored in a band smaller than the estimated value. Encode
The information processing method according to appendix 5, characterized in that:
(Appendix 7)
Processing to input image data;
Processing to input audio data;
A process of encoding the image data; a process of transmitting the encoded image data and the audio data to an information processing apparatus connected via a network;
In the network, the information processing terminal is caused to execute processing for obtaining an estimated value of a bandwidth that can be used for transmission of the image data and the audio data,
The encoding process encodes the image data so as to be within a band smaller than the estimated value.
A program characterized by that.
(Appendix 8)
Causing the information processing terminal to execute processing for detecting a delay in communication via the network;
The encoding process encodes the image data so that it falls within a band smaller than the estimated value when the delay is detected.
The program according to appendix 7, characterized by:
(Appendix 9)
Processing for detecting vibration of the information processing terminal;
Based on the detected vibration, the information processing terminal is caused to execute a process of determining a movement state of the information processing terminal,
The encoding process is performed when the delay is detected, and when the moving state determined in the determination process satisfies a predetermined condition, the image data is stored in a band smaller than the estimated value. Encode
The program according to appendix 8, which is characterized by the above.

DESCRIPTION OF SYMBOLS 10 Worker terminal 20 Supporter terminal 21 Input / output control part 22 Communication control part 23 Audio | voice transmission / reception part 24 Difference image receiving part 25 Image display part 100 Drive apparatus 101 Recording medium 102 Auxiliary storage apparatus 103 Memory apparatus 104 CPU
105 interface device 106 display device 107 input device 108 camera 109 microphone 110 speaker 111 acceleration sensor 121 input / output control unit 122 audio compression unit 123 audio transmission / reception unit 124 audio reproduction unit 125 image display unit 126 display image acquisition unit 127 image difference detection unit 128 Image compression unit 129 Image transmission control unit 130 Wireless communication control unit 131 Delay detection unit 132 Band estimation unit 133 Image quality adjustment unit 134 Moving state determination unit B Bus N1 IP network N2 Wireless communication network

Claims (5)

An information processing terminal,
An image input unit for inputting image data;
A voice input unit for inputting voice data;
An encoding unit for encoding the image data;
A transmission unit for transmitting the encoded image data and the audio data to an information processing apparatus connected via a network;
An estimation unit that obtains an estimated value of a bandwidth that can be used for transmission of the image data and the audio data in the network ;
A second detection unit for detecting vibration of the information processing terminal;
A determination unit that determines a movement state of the information processing terminal based on the detected vibration ;
When the moving state determined by the determination unit satisfies a predetermined condition, the encoding unit encodes the image data so as to be within a band smaller than the estimated value by changing an image quality of the image data. When the moving state determined by the determination unit does not satisfy a predetermined condition, the image data is encoded so as to be within a band smaller than the estimated value by changing a frame rate of the image data.
An information processing terminal characterized by that. A first detection unit that detects a delay in communication via the network;
When the delay is detected, the encoding unit encodes the image data so as to be within a band smaller than the estimated value.
The information processing terminal according to claim 1. The encoding unit encodes the image data so as to be within a band smaller than the estimated value when the delay is detected and the movement state determined by the determination unit satisfies a predetermined condition. ,
The information processing terminal according to claim 2. Information processing terminal
Processing to input image data;
Processing to input audio data;
A process of encoding the image data; a process of transmitting the encoded image data and the audio data to an information processing apparatus connected via a network;
Processing for obtaining an estimated value of a bandwidth that can be used for transmission of the image data and the audio data in the network ;
Processing for detecting vibration of the information processing terminal;
A process of determining a movement state of the information processing terminal based on the detected vibration ;
When the moving state determined in the determining process satisfies a predetermined condition , the encoding process changes the image quality of the image data so that the image data falls within a band smaller than the estimated value. Encode and encode the image data so that it falls within a band smaller than the estimated value by changing the frame rate of the image data when the moving state determined in the determining process does not satisfy a predetermined condition To
An information processing method characterized by the above. In the information processing terminal,
Processing to input image data;
Processing to input audio data;
A process of encoding the image data; a process of transmitting the encoded image data and the audio data to an information processing apparatus connected via a network;
Processing for obtaining an estimated value of a bandwidth that can be used for transmission of the image data and the audio data in the network ;
Processing for detecting vibration of the information processing terminal;
A process for determining a movement state of the information processing terminal based on the detected vibration ;
When the moving state determined in the determining process satisfies a predetermined condition , the encoding process changes the image quality of the image data so that the image data falls within a band smaller than the estimated value. Encode and encode the image data so that it falls within a band smaller than the estimated value by changing the frame rate of the image data when the moving state determined in the determining process does not satisfy a predetermined condition To
A program characterized by that.

Images