JP7485967B2 - Information processing device, control method, and program - Google Patents

Description

The present invention relates to a technology for photographing in cooperation with a device equipped with a photographing means.

Systems for communication between distant locations are becoming widespread. For example, at construction sites, workers on-site can use information processing devices such as smartphones and tablets to carry out work while communicating with operators in remote locations (see, for example, Patent Document 1).

JP 2018-36812 A

However, in a system such as that described in Patent Document 1, depending on the information processing device, the image captured may be of poor quality and may lack visibility, making it difficult to communicate with each other.

To solve these problems, it is conceivable that a highly accurate camera could be connected externally to the information processing terminal and images captured by the camera could be imported. However, when communicating online, exchanging video captured by the camera between the camera and the information processing terminal would impose a communication load between the two devices, which could cause problems in sharing image data.

The present invention aims to provide a mechanism for linking image data obtained by photographing in conjunction with a device equipped with a photographing means.

The present invention, which aims to achieve the above object, is an information processing device that communicates with other devices, and is characterized by comprising: a storage means for storing acquisition information for acquiring image data captured by a device equipped with an external imaging unit with which communication can be performed; and an acquisition means for acquiring image data captured by the device equipped with the external imaging unit based on the acquisition information when communication with the other device is terminated.

The present invention has the advantage that it is possible to link image data obtained by capturing images in cooperation with a device equipped with a capturing means.

FIG. 1 is a diagram showing a schematic configuration of an information processing system. FIG. 2 is a diagram showing a schematic hardware configuration of a client terminal, a server, and an administrator terminal. 13 is a flowchart showing a login process. 13 is a flowchart showing a process in which a user enters a room. 10 is a flowchart showing a process for activating an external camera. 13 is a flowchart showing a recording process. 13 is a flowchart showing a process for downloading recorded data. 13 is a flowchart showing a process for uploading recorded data. FIG. 2 is a diagram showing the configuration of each table etc. FIG. 2 is a diagram showing the configuration of a login screen. FIG. 13 is a diagram showing the configuration of a room list screen. FIG. 13 is a diagram showing the configuration of a communication screen. FIG. 2 is a diagram showing the configuration of a download screen and an upload screen.

The following describes an embodiment of the present invention in detail with reference to the drawings.

The information processing system 100 in FIG. 1 includes a camera 102, which is a device equipped with an imaging means; a client terminal 104, which is an information processing device also equipped with an imaging means; a server 106, which is an information processing device that receives and manages information related to imaging (imaging information) such as images obtained by imaging with the camera 102 or the client terminal 104 and detected sounds; and an administrator terminal 108, which is an information processing device that can display the imaging information managed by the server 106, give operating instructions to the camera 102 and the client terminal 104, and make calls to the client terminal 104.

Next, referring to FIG. 2, an example of a hardware configuration that can be applied to the client terminal 104 in an embodiment of the present invention will be described.

In FIG. 2, 201 is a CPU that performs overall control of each device and controller connected to a system bus 204. In addition, ROM 202 or external memory 211 stores a BIOS (Basic Input/Output System), which is a control program for the CPU 201, an operating system (hereinafter, OS), and various programs (described below) required to realize the functions executed by each server or each PC.

203 is a RAM, which functions as the main memory, work area, etc. of the CPU 201. The CPU 201 loads programs and the like required for executing processing from the ROM 202 or the external memory 211 into the RAM 203, and executes the loaded programs to realize various operations.

In addition, 205 is an input controller that controls input from an input device 209. Examples of the input device 209 include a keyboard, a touch panel, and a pointing device such as a mouse.

When the input device 209 is a touch panel, the user can give various instructions by pressing (touching with a finger, etc.) icons, cursors, or buttons displayed on the touch panel.

The touch panel may also be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.

206 is a video controller that controls the display on a display device such as a display 210. Note that the display 210 refers to a display device such as a CRT or liquid crystal display. Alternatively, it may be another display device.

Note that this also includes the display of a notebook computer that is integrated into the main body. The external output device is not limited to a display, and may be, for example, a projector.

Furthermore, if the device to which the hardware configuration of FIG. 2 is applied is a device capable of accepting the touch operation described above, the display 210 provides the function of the input device 209.

207 is a memory controller that controls access to an external memory 211 such as a hard disk (HDD) that stores boot programs, various applications, font data, user files, edited files, various data, etc., a flexible disk (FD), or a compact flash (registered trademark) memory connected to a PCMCIA card slot via an adapter.

208 is a communication I/F controller that connects and communicates with external devices via a network and executes communication control processing on the network. For example, communication using TCP/IP is possible.

The CPU 201 enables display on the display 210 by, for example, executing an outline font expansion (rasterization) process on a display information area in the RAM 203. The CPU 201 also enables user instructions using a mouse cursor (not shown) on the display 210.

The various programs described below for implementing the present invention are recorded in ROM 202 or external memory 211, and are executed by CPU 201 by being loaded into RAM 203 as necessary.

Furthermore, definition files, various information, tables, etc. used when executing the above programs are also stored in the external memory 211, and detailed explanations of these will be given later.

The camera controller 212 controls the acquisition of images by the camera 214 etc., and the audio controller 213 controls the acquisition of audio by the microphone/speaker 215 etc. and the output to the speaker.

Camera 214 is a camera built into the device to which the hardware configuration in FIG. 2 is applied. Microphone/speaker 215 is a microphone and speaker built into the device to which the hardware configuration in FIG. 2 is applied. This concludes the explanation of FIG. 2.

Note that the server 106 and the administrator terminal 108 have almost the same configuration as the client terminal 104, so their explanation will be omitted.

Next, the process when the client terminal 104 logs in to the information processing system 100 will be described using the flowchart shown in FIG. 3. Note that the process in each step is executed under the control of the CPU in each device.

In step S100, the client terminal 104 displays the login screen 400 (see FIG. 10), and upon detecting that the login button 401 on the login screen 400 has been pressed, acquires login information including the username and password entered in the username 402 and password 403 fields on the login screen 400.

In step S102, the client terminal 104 transmits the login information acquired in step S100 to the server 106.

In step S104, the server 106 receives the login information sent from the client terminal 104 in step S102.

In step S106, the server 106 checks whether the login information received in step S104 exists in the user authentication information table 300 (see FIG. 9).

As shown in the upper part of Figure 9, the user authentication information table 300 contains information about the user name, password, and authority to use the system.

In this step, it is checked whether the username and password contained in the login information match the username and password in the user authentication information table.

In step S108, if the server 106 determines that the login information exists in the user authentication information table 300, it determines that the user can log in to the system and proceeds to step S110; if it does not determine that the login information exists, it proceeds to step S116.

In step S110, if the server 106 can obtain from the user authentication information table 300 the authorization information corresponding to the user name and password in the user authentication information table 300 that match the user name and password in the login information, the server 106 proceeds to step S114; if not, the server 106 proceeds to step S116.

By providing authority information, it is possible to separate the authority for applications that can be used by users on the client terminal 104 from the authority for applications that can be used by operators on the administrator terminal 108.

In step S114, the server 106 obtains room list information from the room information table 302 (see FIG. 10).

As shown in the center of FIG. 10, the room information table 302 is composed of a room ID for uniquely identifying the room, a room name indicating the name of the room, the number of occupants indicating the number of people in the room, external camera usability indicating whether an external camera can be connected and used, the maximum number of people allowed in the room, and occupant users indicating the users who have entered the room.

In step S116, the server 106 obtains error information indicating that the system cannot be used.

In step S118, the server 106 transmits the room list information acquired in step S114 or the error information acquired in step S116 to the client terminal 104 as login result information.

In step S120, the client terminal 104 receives the login result information sent from the server 106 in step S118.

In step S122, the client terminal 104 determines whether the login result information received in step S120 includes error information, and if it determines that the error information includes error information, the process proceeds to step S124, and if it determines that the error information does not include error information, the process proceeds to step S126.

In step S124, the client terminal 104 displays an error screen (not shown) indicating that the system cannot be used based on the error information.

In step S126, the client terminal 104 displays information about the rooms on the room list screen 500 (see FIG. 11) based on the room list information in the login result information.

The room list screen 500 includes a room name 501 indicating the name of the room, the number of people currently in the room 502, restrictions 503 indicating usage restrictions such as whether an externally connected camera can be used in the room or only the built-in camera can be used, and an enter button 504 to be pressed when entering the room.

Next, the process of a user entering a room will be explained using the flowchart shown in FIG. 4. Note that the process in each step is executed under the control of the CPU in each device.

In step S200, the client terminal 104 acquires information (room ID, etc.) about the room for which the enter button 504 was pressed on the room list screen 500.

In step S202, the client terminal 104 transmits the information about the room acquired in step S200 to the server 106.

In step S204, the server 106 receives information about the room sent from the client terminal 104 in step S202.

In step S206, the server 106 obtains a record in the room information table 302 that corresponds to the information about the room received in step S204 (e.g., the room ID matches).

In step S208, the server 106 increments the number of occupants in the record acquired in step S206 by one, and determines whether or not the number is within the allowable number of occupants. If it is determined that the number is within the allowable number of occupants, the process proceeds to step S210. If it is not determined that the number is within the allowable number of occupants, the server 106 determines that the room is full and proceeds to step S212.

In step S210, the server 106 allows the user to enter the room, and is configured to be able to communicate with the terminals of users participating in the room. The server 106 also registers the user name of the entering user in the room information table.

In step S212, the server 106 obtains error information indicating that entry to the room is not possible.

In step S214, the server 106 transmits information indicating that entry was permitted in step S210 or the error information obtained in step S212 to the client terminal 104 as room entry result information.

In step S216, the client terminal 104 receives the entry result information sent from the server 106 in step S214.

In step S218, the client terminal 104 determines whether the entry result information received in step S216 is error information, and if it is determined that the information is error information, the process proceeds to step S232, and if it is not determined that the information is error information, the process proceeds to step S220.

In step S220, the client terminal 104 determines whether the built-in camera is set by default to be available at startup, and if it is not set, the process proceeds to step S222; if it is not set, the process proceeds to step S224.

In step S222, the client terminal 104 determines whether or not the room is restricted from activating the camera 102 connected so as to be able to communicate with the outside, based on the information displayed in the restriction 503. If it is determined that the room is restricted, the process proceeds to step S224, and if it is not determined that the room is restricted, the process proceeds to step S226.

In step S224, the client terminal 104 activates the built-in camera, and in step S226, the client terminal 104 activates the external camera 102 that is connected so as to be capable of communication (described later in FIG. 5).

The external camera 102 available in the room may be one preset camera, or a list of the external cameras 102 may be displayed on a screen (not shown) and one may be selected and activated.

In step S228, the client terminal 104 determines whether or not an error notification was issued when the camera was started, and if it is determined that an error notification was not issued, the process proceeds to step S230, and if it is determined that an error notification was not issued, the process proceeds to step S232.

In step S230, the client terminal 104 displays the communication screen 600 (see FIG. 12) and starts communication with the administrator terminal 108 used by the operator.

Figure 12 shows the configuration of the communication screen 600, which is configured to display image data obtained by capturing images with the built-in camera or image data obtained by capturing images with the external camera 102.

In addition, information identifying whether the image data was obtained by capturing an image using the built-in camera or the external camera 102 may be displayed.

Alternatively, when displaying image data obtained by capturing images using an external camera 102, specific information may be displayed that identifies the camera 102 that captured the displayed image data from among several external cameras 102.

Next, the process of starting up the external camera 102 that is connected so as to be capable of communication will be described using the flowchart shown in FIG. 5. Note that each step in this process is executed under the control of the CPU of the client terminal 104.

In step S300, information for communicating with the external camera 102 to be connected is obtained from a predefined configuration file (not shown), and in step S302, the information obtained in step S400 is sent as connection request information to the camera 102 to be connected.

In step S304, if it is determined that there is a response from the camera 102 that sent the connection request information in step S302, the process proceeds to step S306, and if it is not determined that there is a response (for example, due to a poor connection or the external camera not being powered on), the process proceeds to step S308.

In step S306, it is determined whether or not the camera 102 has been started based on the response received from the camera 102 in step S304. If it is determined that the camera 102 has been started, the process proceeds to step S310. If it is determined that the camera 102 has not been started, the process proceeds to step S308.

In step S308, if there is no response after a predetermined time has elapsed in step S304, or if startup has failed in step S306, error information indicating this is sent, and in step S310, a notification is sent that startup was successful.

Next, the process of recording in the camera 102 will be explained using the flowchart shown in FIG. 6. Note that each step in this process is executed under the control of the CPU of the client terminal 104.

In step S400, when the record button 606 on the communication screen 600 is pressed, a record request packet is created to instruct the camera 102 to start recording and is sent to the camera 102.

When the capture button 604 on the communication screen 600 is pressed, a capture instruction is sent to the camera 102, and the camera 102 captures a still image, allowing the camera 102 to obtain the image.

In step S402, it is determined whether there has been a response from the camera 102 to the recording request packet sent in step S400. If there has been a response, the process proceeds to step S404. If there has been no response, the process proceeds to step S408.

In step S404, it is determined whether the camera 102 has successfully started recording. If recording has started successfully, the process proceeds to step S406. If recording has not started successfully, the process proceeds to step S408.

In step S406, a download ticket is issued indicating that recorded data is available on the camera 102 (see Figure 9).

The lower part of Figure 9 shows the configuration of the download ticket, which includes a ticket number for uniquely identifying the download ticket, a file name (included in the response of step S402) indicating the file name of the image data (hereinafter, recorded data) obtained by recording on camera 102, a file path (included in the response of step S402) indicating the location where the recorded data is stored on camera 102, a room name indicating the name of the room currently entered, and the time when the recording was made.

In step S408, if there is no response in step S402 due to a poor connection or the camera 102 not being powered on, or if recording has failed to start in step S404, an error is notified.

Next, the process of downloading recorded data stored in the camera 102 to the client terminal 104 will be described using the flowchart shown in FIG. 7. Note that each step in this process is executed under the control of the CPU of the client terminal 104.

In step S500, when the download screen 700 (see FIG. 13) is launched, the download ticket information in the client terminal 104 is obtained.

In step S502, it is determined whether a download ticket exists as a result of obtaining the download ticket information in step S500. If it is determined that a download ticket exists, the process proceeds to step S504. If it is determined that a download ticket does not exist, the process ends.

In step S504, the information of the download ticket acquired in step S502 is displayed on the download screen 700 as a recording ticket.

The download screen 700 includes a number for uniquely identifying the recording ticket, a file name indicating the file name of the recording data to be downloaded using the recording ticket, and the recording time of the recording data.

These items correspond to the download ticket number, file name, and recording time, in the order listed above.

In step S506, the storage location of the recording ticket selected from the recording tickets displayed on the download screen 700 is obtained from the file path of the download ticket.

When obtaining the data, the download ticket number is searched for using the download screen 700 number as the primary key to obtain the file path of the corresponding record. In step S508, a download request packet is created and transmitted to the camera 102, requesting the download of the recorded data, based on information such as the file path and file name of the recorded data obtained in step S506.

In step S510, it is determined whether or not a response has been received from the camera 102. If a response has been received, processing proceeds to step S512. If it is not determined that a response has been received, processing proceeds to step S526.

In step S512, the recorded data stored in the camera 102 is downloaded, and in step S514, when the download of the recorded data is complete, a download completion packet is created to notify the camera 102 of the completion of the download, and the packet is sent to the camera 102.

In step S516, it is determined whether or not there has been a response from the camera 102 in response to the notification of download completion in step S512. If it is determined that there has been a response, processing proceeds to step S518; if it is determined that there has not been a response, processing proceeds to step S526.

In step S518, a recording data deletion request packet is created and transmitted, requesting the camera 102 to delete the recording data that has been downloaded.

In step S520, it is determined whether or not there has been a response from the camera 102. If it is determined that there has been a response, the process proceeds to step S522. If it is determined that there has not been a response, the process proceeds to step S526.

In step S522, the download ticket for the downloaded recording data is deleted, and in step S524, a notification is sent that the download of the recording data has been completed.

In step S526, if there is no response due to a poor connection in step S510 or because the camera 102 is not turned on, or if the download or deletion of recorded data in step S516 or step S520 has failed, an error is notified.

Next, the process of uploading the recording data downloaded to the client terminal 104 to the server 106 will be described using the flowchart shown in FIG. 8. This process is executed under the control of the CPU of the client terminal 104 or the server 106.

In step S600, when the upload screen 800 (see FIG. 13) is launched, the client terminal 104 obtains information about the downloaded recording data in the client terminal 104 and displays it on the upload screen 800.

For the recorded data selected from the list of recorded data displayed on the upload screen 800, a packet is created and sent to the server 106 for uploading.

In step S602, the server 106 receives the recording data upload packet sent in step S600.

The download screen 700 and the upload screen 800 are displayed by pressing the download screen display tab 702 and the upload screen display tab 802, respectively.

In step S604, the server 106 determines whether or not the recording data was successfully received. If it is determined that the recording data was successfully received, the process proceeds to step S608. If it is determined that the recording data was not successfully received, the process proceeds to step S606.

In step S606, if uploading of the recorded data fails, the server 106 obtains information about the error.

In step S608, the server 106 creates a reply packet including information on whether the upload of the recorded data was successful and sends it to the client terminal 104.

In step S610, the client terminal 104 receives the reply packet sent from the server 106 in step S608.

In step S612, the client terminal 104 determines whether the upload of the recorded data was successful based on the reply packet received in step S610. If it is determined that the upload was successful, the process proceeds to step S614. If it is determined that the upload was not successful, the process proceeds to step S616.

In step S614, the client terminal 104 deletes the recording data that was successfully uploaded, and in step S616, the client terminal 104 notifies the client of the error based on the information about the error.

The administrator terminal 108 makes it possible to refer to the recorded data uploaded to the server 106, and to view clearer image data.

As described above, it goes without saying that the object of the present invention can be achieved by supplying a recording medium on which a program that realizes the functions of the above-mentioned embodiments is recorded to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

In this case, the program read from the recording medium itself realizes the novel functions of the present invention, and the recording medium on which the program is recorded constitutes the present invention.

Recording media for supplying the program may include, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a DVD-ROM, a magnetic tape, a non-volatile memory card, a ROM, an EEPROM, a silicon disk, etc.

Furthermore, it goes without saying that not only are the functions of the above-mentioned embodiments realized by the computer executing a program it has read, but also that the functions of the above-mentioned embodiments can be realized by an operating system (OS) or the like running on the computer carrying out some or all of the actual processing based on the instructions of the program.

Furthermore, it goes without saying that this also includes cases where a program read from a recording medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to a computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.

The present invention can be applied to a system consisting of multiple devices, or to a device consisting of a single device.

It goes without saying that the present invention can also be applied to cases where the invention is achieved by supplying a program to a system or device. In this case, by reading a recording medium that stores a program for achieving the invention into the system or device, the system or device can enjoy the effects of the present invention.

Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention.

All configurations that combine the above-mentioned embodiments and their modified examples are also included in the present invention.

100 Information processing system 102 Camera 104 Client terminal 106 Server 108 Administrator terminal

Claims (11)

A communication means for communicating with other devices;
an acquisition means for acquiring, based on completion of communication with another device by the communication means, image data captured by an external imaging device during the communication, based on acquisition information for acquiring the image data ;
Equipped with
The information processing apparatus according to claim 1, wherein the acquired information is information that is issued when imaging by the external imaging device is started . A communication means for communicating with other devices;
an acquisition means for acquiring image data captured by an external imaging device during communication with another device based on the termination of the communication with the communication means;
a control means for controlling, when the image data is acquired by the acquisition means, to delete the image data stored in the external imaging device that captured the image data;
An information processing device comprising: 3. The information processing apparatus according to claim 2, wherein the control means performs control so that, when the image data is acquired by the acquisition means, acquisition information for acquiring the image data is deleted. an entrance means for entering a room for communicating with other devices via a communication means;
A communication means for communicating with other devices;
an acquisition means for acquiring image data captured by an external imaging device during communication with another device based on the termination of the communication with the communication means;
Equipped with
The room is characterized in that information on whether or not to permit the use of an external imaging device is set,
The information processing apparatus further comprises a requesting means for requesting the external imaging device to start imaging when the room entered by the entering means is a room in which use of the external imaging device is permitted. 5. The information processing apparatus according to claim 1, further comprising a storage unit for storing acquisition information for acquiring image data by said acquisition unit. 6. The information processing apparatus according to claim 5, wherein the acquired information includes a file name of the image data. 7. The information processing apparatus according to claim 5, wherein the acquired information includes information indicating a storage destination of the image data. A communication step in which a communication means of the information processing device communicates with other devices;
an acquisition step in which an acquisition means of the information processing device acquires, based on completion of communication with another device through the communication step, image data captured by an external imaging device while the communication is being performed, based on acquisition information for acquiring the image data;
Equipped with
The information processing method according to claim 1, wherein the acquired information is information that is issued when imaging by the external imaging device is started. A communication step in which a communication means of the information processing device communicates with other devices;
an acquisition step in which an acquisition means of the information processing device acquires image data captured by an external imaging device while the communication with another device is being performed based on the completion of the communication with the other device through the communication step;
a control step in which, when image data is acquired by the acquisition step, a control means of the information processing device controls the information processing device to delete the image data stored in the external imaging device that captured the image data;
An information processing method comprising: an entrance step in which an entrance means of the information processing device enters a room for communication with other devices;
a communication step in which a communication means of the information processing device communicates with other devices;
an acquisition step in which an acquisition means of the information processing device acquires image data captured by an external imaging device while the communication with another device is being performed based on the completion of the communication with the other device through the communication step;
Equipped with
The room is characterized in that information on whether or not to permit the use of an external imaging device is set,
The information processing method according to the present invention further comprises a request step of requesting the external imaging device to start capturing images when the room entered by the entry means is a room in which use of an external imaging device is permitted. A program for causing a computer to function as each of the means according to any one of claims 1 to 7 .

Images