JP7415376B2 - Information processing device, information processing method, program - Google Patents

Description

The present technology relates to an information processing device, an information processing method, and a program, and particularly relates to a technology for editing metadata added to image data.

For users who use imaging devices (also called "cameras") for work, such as professional photographers and journalists, images captured by the imaging device can be sent to a server at a newspaper company or other such site using the communication function of the imaging device. The data is uploaded to an FTP (File Transfer Protocol) server (for example, Patent Document 1).
Further, in newspaper companies and the like, image data is uploaded by multiple users, so the amount of image data received can be enormous. Newspaper companies and the like utilize metadata added to image data to efficiently search for desired image data.
To this end, it is conceivable that the user performs an operation on the imaging device to add metadata to image data.

Japanese Patent Application Publication No. 2017-138777

However, the input means provided in the imaging device is not suitable for inputting text for adding metadata, and it is difficult to quickly upload captured images.

Therefore, the present technology aims to provide an environment suitable for editing metadata for users, such as professional photographers, who need to add metadata to captured images and quickly upload them.

An information processing apparatus according to the present technology includes a user interface control unit that performs a process of detecting an on/off switching operation for displaying each field of metadata added to an image.
This makes it possible to display only those fields included in the metadata that the user wants to edit.

The user interface control unit in the information processing apparatus described above may perform a process of controlling whether or not the metadata field can be displayed based on the detected switching operation.
Thereby, the result of the user's switching operation is reflected on the display.

The above-described information processing apparatus may include a storage unit that stores the display permission/prohibition setting.
Thereby, the result of the user's switching operation is saved and used for display.

The user interface control unit in the information processing apparatus described above may perform a process of detecting an operation for selecting a field to be edited from among the fields of the metadata displayed based on the display availability.
This allows the user to edit the item to be edited.

The user interface control unit in the information processing apparatus described above may perform user interface processing that allows editing of display names for each field.
This makes it possible to perform an operation to give a different display name when the field name of metadata is difficult to understand.

The user interface control unit in the information processing apparatus described above may perform the user interface processing so that field names cannot be edited.
This prevents, for example, a field name determined by the specifications from being changed and image file classification processing based on metadata not functioning properly.

The user interface control unit in the information processing device described above may perform a process of detecting an operation to edit text information as the metadata.
This allows the user to edit the text information of the metadata.

The information processing device described above may be a mobile terminal device.
Images captured by an imaging device can be transferred to a mobile terminal device serving as an information processing device on the spot, and metadata can be edited quickly.

In the information processing device described above, the metadata may be IPTC metadata.
By using metadata formatted by an organization or the like, the above-mentioned technology can be widely applied.

In the information processing method of the present technology, an information processing apparatus executes a process of detecting an on/off switching operation for displaying each field of metadata added to an image.

The program of the present technology is a program that causes an information processing apparatus to execute a process of detecting an on/off switching operation for displaying each field of metadata added to an image.
This allows the information processing device to execute various processes for providing an environment suitable for editing metadata.

FIG. 2 is an explanatory diagram of transfer and upload of image files and audio files according to the embodiment of the present technology. FIG. 2 is a block diagram of an imaging device that performs communication in an embodiment. FIG. 3 is an explanatory diagram of IPTC metadata added to image data in the embodiment. FIG. 1 is a block diagram of an information processing device according to an embodiment. FIG. 2 is an explanatory diagram of the functional configuration of the information processing device according to the embodiment. FIG. 3 is an explanatory diagram of an image list screen according to the embodiment. FIG. 3 is an explanatory diagram of a menu screen according to the embodiment. It is an explanatory diagram of a setting screen of an embodiment. FIG. 3 is an explanatory diagram of a metadata editing setting item screen according to the embodiment. It is an explanatory diagram of an IPTC field setting screen of an embodiment. FIG. 3 is an explanatory diagram of a field display name change dialog according to the embodiment. It is an explanatory diagram of an IPTC field setting screen of an embodiment. It is an explanatory diagram of an IPTC field setting screen of an embodiment. FIG. 3 is an explanatory diagram of a caption editing screen according to the embodiment. FIG. 6 is an explanatory diagram of a caption editing screen with voice memo text added to caption data in the embodiment. FIG. 3 is an explanatory diagram of an IPTC metadata editing screen according to the embodiment. FIG. 3 is an explanatory diagram of an IPTC metadata editing screen according to the embodiment. It is an explanatory view of a voice memo automatic caption addition setting screen of an embodiment. FIG. 3 is an explanatory diagram of an automatic upload setting OFF state of an automatic upload setting screen according to the embodiment. FIG. 6 is an explanatory diagram of the automatic upload setting screen in the embodiment when the automatic upload setting is on; FIG. 6 is an explanatory diagram of the automatic upload setting ON state of the automatic upload setting screen according to the embodiment. 12 is a flowchart of an example of processing when capturing an image according to the embodiment. 12 is a flowchart of an example of processing when capturing an image according to the embodiment. 3 is a flowchart of upload file preparation processing according to the embodiment. 12 is a flowchart of an example of processing from an image list screen according to the embodiment. 12 is a flowchart of an example of processing from an image list screen according to the embodiment. 12 is a flowchart of an example of processing from an image list screen according to the embodiment. 12 is a flowchart of an example of processing at the time of uploading according to the embodiment. 12 is a flowchart of a processing example regarding the IPTC field setting screen according to the embodiment.

Hereinafter, embodiments will be described in the following order.
<1. Importing and uploading images in the embodiment>
<2. Configuration of imaging device>
<3. Configuration of information processing device>
<4. User interface screen>
<5. Example of processing when importing images>
<6. Processing example including text conversion and caption editing>
<7. Example of processing when uploading>
<8. Processing related to IPTC field setting screen>
<9. Summary and modifications>
<10. This technology＞

<1. Importing and uploading images in the embodiment>
FIG. 1 shows an imaging device 1 including an information processing device 2 of this embodiment, an FTP server 4, a text conversion engine 5, and a network 6.

The imaging device 1 includes various imaging devices such as a video camera and a still camera. The illustrated imaging device 1 is assumed to be a camera used by photographers and reporters at sports and event venues, reporting sites, and the like.

The information processing device 2 is exemplified here by a portable terminal device such as a smartphone.
Specific examples of information processing devices include various examples such as personal computers, tablet information processing devices, mobile phone devices, game devices, audio devices, video devices, communication devices, television devices, and server devices. be done. Any device capable of performing calculations as information processing, for example, a device incorporating a microcomputer, can be implemented as the information processing device of the present disclosure.
However, if the device is intended to be used at an event venue or the like as described above, it is preferable to use a mobile terminal such as a smartphone or a tablet device.

The imaging device 1 and the information processing device 2 communicate information with each other using, for example, Bluetooth (registered trademark), WI-FI (registered trademark) communication, near field communication such as NFC (Near Field Communication), or infrared communication. It is considered possible.
Note that the imaging device 1 and the information processing device 2 may be able to communicate with each other through wired connection communication.
Further, the information processing device 2 functions as an FTP server, the imaging device 1 functions as an FTP client, and image data and the like are uploaded from the imaging device 1 to the information processing device 2. That is, the imaging device 1 stores FTP setting information for performing FTP transfer, and by using the FTP setting information, the FTP transfer is performed using the information processing device 2 as an FTP server.

In the case of this embodiment, a situation is assumed in which an image file PF and an audio file AF are transmitted from the imaging device 1 to the information processing device 2.
The information processing device 2 holds the image file PF and audio file AF transferred from the imaging device 1, and can present them to the user, add and edit metadata, and upload them to the FTP server 4. .

The imaging device 1 generates image data as a still image or a moving image through an imaging operation. Note that the metadata can also be added by the imaging device 1.
It is assumed that the image file PF shown in FIG. 1 is a data file containing this image data and metadata.

The imaging device 1 has a voice memo function. This is a function that allows the user to add annotations, explanations, etc. to the captured image by voice by inputting voice when capturing the image. For example, when a single still image is captured, the cameraman can make a sound to explain the content of the image while performing certain operations, or say something while specifying the image, and the sound is recorded and the image is It is considered to be a voice memo associated with the data.
It is assumed that the audio file AF shown in FIG. 1 is a data file containing audio data as this voice memo.
Note that when capturing a moving image, ambient sound is also recorded as audio track data, but the audio track data is audio data included in the image file PF and is separate from the audio file AF. In the explanation, the audio file AF refers to a file containing audio data as a voice memo.
In the following description, assuming still image capturing, an example will be explained in which the image file PF includes still image data and metadata, and the audio file AF includes voice memo data generated in conjunction with still image capturing.

Note that the audio file AF is not necessarily associated with every image file PF, and the audio file AF is generated by the imaging device 1 only when a cameraman or the like inputs audio using the voice memo function. Associated with image file PF.
Therefore, in data transfer from the imaging device 1 to the information processing device 2, there are cases where the image file PF and the audio file AF are sent as a pair, and cases where only the image file PF is sent.

The information processing device 2 can upload the transferred image file PF and audio file AF to the FTP server 4 via the network 6.
The network 6 is assumed to be, for example, the Internet, a home network, a LAN (Local Area Network), a satellite communication network, or other various networks.
The FTP server 4 may be, for example, a server operated by a newspaper company, broadcasting station, communication company, or the like. Of course, it is not limited to such servers.
The FTP server 4 is assumed to be a cloud server, a home server, a personal computer, or the like.

The information processing device 2 not only simply uploads the image file PF etc. from the imaging device 1 to the FTP server 4, but also performs operations such as adding and editing caption data included in metadata, setting the image size, and compressing the data. You can then upload it.
Furthermore, for the image file PF associated with the audio file AF, the information processing device 2 also acquires the audio data in the audio file AF, that is, the text data obtained by converting the above-mentioned voice memo into text, and adds it to the metadata. You can then upload it.

In particular, in the case of this embodiment, the information processing device 2 performs interface control to facilitate editing of metadata.
Specifically, as will be described later, it provides a function to change the display name of each item as metadata to an easy-to-use name, and a function to display only an arbitrary item from a plurality of items as an editing target.

It is conceivable that the information processing device 2 is equipped with a text conversion engine to convert the voice memo into text data, but the information processing device 2 itself is not equipped with a text conversion function, and external text A conversion engine 5 may also be used.
For example, the information processing device 2 transmits audio data of a voice memo to the text conversion engine 5 via the network 6.
The text conversion engine 5 performs processing to convert this audio data into text, and sends the generated text data to the information processing device 2. This allows the information processing device 2 to obtain text data obtained by converting the voice memo into text.
For the sake of explanation, text data obtained by converting a voice memo into text will also be referred to as "voice memo text."

By constructing the communication system as shown in FIG. 1, for example, when the user of the imaging device 1 is a professional photographer working for a newspaper company, the following things become possible.
That is, the photographer transfers the image taken at the event venue to his/her smartphone (information processing device 2). It is assumed that the system will be used in such a manner that the captured image is uploaded from the information processing device 2 to the FTP server 4 after performing necessary operations such as caption editing on the information processing device 2 such as a smartphone. Further, after the image is transferred to the information processing device 2 such as a smartphone, the information processing device 2 may automatically upload the image to the FTP server 4.
In such an embodiment, the voice memo is converted into text, added to metadata, and uploaded together with image data.

<2. Configuration of imaging device>
An example of the configuration of the imaging device 1 will be explained with reference to FIG.
The imaging device 1 includes, for example, a lens system 11, an image sensor section 12, a camera signal processing section 13, a recording control section 14, a display section 15, a communication section 16, an operation section 17, a camera control section 18, a memory section 19, and a driver section 22. , a sensor section 23, an audio input section 25, and an audio processing section 26.

The lens system 11 includes lenses such as a zoom lens and a focus lens, an aperture mechanism, and the like. This lens system 11 guides light (incident light) from the subject and focuses it on the image sensor section 12 .

The image sensor section 12 includes, for example, an image sensor 12a (image sensor) such as a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type.
The image sensor unit 12 performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, etc. on the electrical signal obtained by photoelectrically converting the light received by the image sensor 12a, and further performs A/D (Analog/Digital) Perform conversion processing. The imaging signal as digital data is then output to the camera signal processing section 13 and camera control section 18 at the subsequent stage.

The camera signal processing unit 13 is configured as an image processing processor using, for example, a DSP (Digital Signal Processor). This camera signal processing section 13 performs various signal processing on the digital signal (captured image signal) from the image sensor section 12. For example, as a camera process, the camera signal processing unit 13 performs preprocessing, synchronization processing, YC generation processing, resolution conversion processing, file formation processing, and the like.

In the preprocessing, clamping processing for clamping the R, G, and B black levels to predetermined levels, and correction processing between the R, G, and B color channels are performed on the captured image signal from the image sensor unit 12. conduct.
In the simultaneous processing, color separation processing is performed so that the image data for each pixel includes all R, G, and B color components. For example, in the case of an image sensor using a Bayer array color filter, demosaic processing is performed as color separation processing.
In the YC generation process, a luminance (Y) signal and a color (C) signal are generated (separated) from R, G, and B image data.
In the resolution conversion process, the resolution conversion process is performed on image data that has been subjected to various types of signal processing.

In the file formation process, for example, the image data that has been subjected to the various processes described above is compressed and encoded for recording and communication purposes, formatting, and generation and addition of metadata are performed to generate files for recording and communication purposes. I do.
For example, an image file PF in a format such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), or GIF (Graphics Interchange Format) is generated as a still image file. It is also conceivable to generate an image file PF in the MP4 format, which is used for recording video and audio in accordance with MPEG-4.
Note that it is also possible to generate the image file PF as raw (RAW) image data.

Regarding metadata, the camera signal processing section 13 includes information on processing parameters within the camera signal processing section 13, various control parameters acquired from the camera control section 18, and information indicating the operating states of the lens system 11 and the image sensor section 12. , mode setting information, and imaging environment information (date, time, location, etc.).
In particular, in the case of this embodiment, IPTC (International Press Telecommunications Council) metadata illustrated in FIG. 3 is included.
IPTC metadata is metadata in a format established by a media company organization, and Figure 3 shows only some of its items as examples, such as "description/caption", "description writer", "headline", " It is possible to write a variety of information such as "keywords".

The recording control unit 14 performs recording and reproduction on a recording medium such as a nonvolatile memory. The recording control unit 14 performs processing for recording image files such as moving image data and still image data, thumbnail images, etc. onto a recording medium, for example.
The actual form of the recording control section 14 can be considered in various ways. For example, the recording control unit 14 may be configured as a flash memory built into the imaging device 1 and its writing/reading circuit. The recording control unit 14 may also be a card recording/reproducing unit that performs recording/reproducing access to a recording medium that can be attached to and detached from the imaging device 1, such as a memory card (portable flash memory, etc.). Further, the recording control unit 14 may be implemented as a built-in form of the imaging device 1, such as an HDD (Hard Disk Drive).

The display unit 15 is a display unit that provides various displays to the imager, and includes, for example, a display such as a liquid crystal display (LCD) or an organic EL (electro-luminescence) display disposed on the housing of the imaging device 1. It is considered to be a display panel or viewfinder depending on the device.
The display unit 15 executes various displays on the display screen based on instructions from the camera control unit 18.
For example, the display unit 15 displays a reproduced image of the image data read from the recording medium by the recording control unit 14.
Further, the display section 15 is supplied with image data of the captured image whose resolution has been converted for display by the camera signal processing section 13, and the display section 15 is supplied with the image data of the captured image based on the image data of the captured image in accordance with instructions from the camera control section 18. may be displayed. As a result, a so-called through image (monitoring image of the subject), which is an image captured while checking the composition or recording a video, is displayed.
Further, the display unit 15 causes various operation menus, icons, messages, etc., that is, displays as a GUI (Graphical User Interface), to appear on the screen based on instructions from the camera control unit 18.

The communication unit 16 performs wired or wireless data communication and network communication with external devices.
For example, captured image data (still image files and video files) is transmitted and output to an external display device, recording device, playback device, etc.
Further, the communication unit 16 serves as a network communication unit, and performs communication using various networks 6 such as the Internet, a home network, and a LAN (Local Area Network), and sends and receives various data to and from servers, terminals, etc. on the network. I can do it. For example, although not shown in FIG. 1, the communication unit 16 may perform communication processing to upload captured image data (such as the above-mentioned image file) to the FTP server 4.
Further, in the case of the present embodiment, the communication unit 16 communicates with the information processing device 2 and transfers the image file PF and the audio file AF.

The operation unit 17 collectively represents input devices through which the user performs various operation inputs. Specifically, the operation unit 17 indicates various operators (keys, dials, touch panels, touch pads, etc.) provided on the casing of the imaging device 1.
The operation unit 17 detects a user's operation, and a signal corresponding to the input operation is sent to the camera control unit 18.

The camera control section 18 is constituted by a microcomputer (arithmetic processing unit) equipped with a CPU (Central Processing Unit).
The memory unit 19 stores information and the like used by the camera control unit 18 for processing. The illustrated memory section 19 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, and the like.
The memory section 19 may be a memory area built into a microcomputer chip as the camera control section 18, or may be constituted by a separate memory chip.
The camera control unit 18 controls the entire imaging device 1 by executing programs stored in the ROM, flash memory, etc. of the memory unit 19.
For example, the camera control unit 18 controls the shutter speed of the image sensor unit 12, instructs various signal processing in the camera signal processing unit 13, performs imaging operations and recording operations in response to user operations, reproduces recorded image files, and performs recording. It controls the operations of each necessary part, such as the transfer operation and upload operation of the image, the operation of the lens system 11 such as zoom, focus, and aperture adjustment in the lens barrel, and the operation of the user interface.

The RAM in the memory unit 19 is used to temporarily store data, programs, etc. as a work area when the CPU of the camera control unit 18 processes various data.
The ROM and flash memory (non-volatile memory) in the memory unit 19 contain an OS (Operating System) for the CPU to control each part, content files such as image files, application programs for various operations, and firmware. , used to store various setting information, etc.
Various setting information includes the above-mentioned FTP setting information, exposure settings, shutter speed settings, and mode settings as setting information related to imaging operations, and white balance settings, color settings, and image effects as setting information related to image processing. There are custom key settings and display settings as setting information related to settings and operability.

The driver section 22 is provided with, for example, a motor driver for a zoom lens drive motor, a motor driver for a focus lens drive motor, a motor driver for an aperture mechanism motor, and the like.
These motor drivers apply drive current to the corresponding drivers in response to instructions from the camera control unit 18 to move the focus lens and zoom lens, open and close the aperture blades of the aperture mechanism, and so on.

The sensor section 23 comprehensively represents various sensors installed in the imaging device.
The sensor unit 23 is equipped with, for example, an IMU (inertial measurement unit), which detects angular velocity with a three-axis angular velocity (gyro) sensor of pitch, yaw, and roll, and detects acceleration with an acceleration sensor. be able to.
Further, as the sensor section 23, for example, a position information sensor, an illuminance sensor, a proximity sensor, etc. may be installed.

The audio input section 25 includes, for example, a microphone and a microphone amplifier, and outputs an audio signal obtained by collecting ambient sounds.
The audio processing unit 26 performs processing for converting the audio signal obtained by the audio input unit 25 into a digital audio signal, AGC processing, sound quality processing, noise reduction processing, and the like. The audio data that has undergone these processes is output to the camera signal processing section 13 and camera control section 18.
For example, when capturing a moving image, the audio data is processed by the camera control unit 18 as audio data accompanying the moving image.

Further, the audio data can be converted into a file as an audio file AF in the camera signal processing unit 13 or the camera control unit 18 as audio data as a so-called voice memo during image capturing.
The audio file AF can be recorded on a recording medium in association with the image file by the recording control section 14, or can be transmitted and output from the communication section 16 together with the image file.

<3. Configuration of information processing device>
FIG. 4 shows a configuration example of the information processing device 2, which is, for example, a mobile terminal device.
The CPU 71 of the information processing device 2 executes various processes according to programs stored in the ROM 72 or programs loaded into the RAM 73 from the storage section 79. The RAM 73 also appropriately stores data necessary for the CPU 71 to execute various processes.
The CPU 71, ROM 72, and RAM 73 are interconnected via a bus 74. An input/output interface 75 is also connected to this bus 74.

The input/output interface 75 is connected to an input section 76 consisting of an operator or an operating device.
For example, as the input unit 76, various operators and operating devices such as a keyboard, a mouse, a key, a dial, a touch panel, a touch pad, and a remote controller are assumed.
A user's operation is detected by the input unit 76, and a signal corresponding to the input operation is interpreted by the CPU 71.

Further, a display section 77 consisting of an LCD or an organic EL panel, and an audio output section 78 consisting of a speaker etc. are connected to the input/output interface 75 either integrally or separately.
The display unit 77 is a display unit that performs various displays, and is configured by, for example, a display device provided in the casing of the information processing device 2, a separate display device connected to the information processing device 2, or the like.
The display unit 77 displays images for various image processing, moving images to be processed, etc. on the display screen based on instructions from the CPU 71. Further, the display unit 77 displays various operation menus, icons, messages, etc., ie, as a GUI (Graphical User Interface), based on instructions from the CPU 71.

The input/output interface 75 may be connected to a storage section 79 made up of a hard disk, a solid state memory, etc., and a communication section 80 made up of a modem or the like.

The communication unit 80 performs communication processing via a transmission path such as the Internet, and communicates with various devices by wire/wireless communication, bus communication, etc.
In the case of this embodiment, the communication unit 80 communicates with the imaging device 1 by, for example, the above-mentioned FTP communication, short-range wireless communication such as Bluetooth, WI-FI, NFC, infrared communication, wired connection communication, etc. have the functionality to do so.

A drive 82 is also connected to the input/output interface 75 as necessary, and a removable recording medium 81 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory is appropriately loaded.
The drive 82 can read data files such as image files and various computer programs from the removable recording medium 81. The read data file is stored in the storage section 79, and images and sounds included in the data file are outputted on the display section 77 and the audio output section 78. Further, computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as necessary.

The information processing device 2 may be equipped with a processor as the text conversion engine 83.
The text conversion engine 83 performs a process of analyzing audio data and converting it into text data, for example.
Note that when using the external text conversion engine 5 as described above, the information processing device 2 does not need to include a processor as the text conversion engine 83.

In this information processing device 2, for example, software for the processing of the present disclosure can be installed via network communication by the communication unit 80 or the removable recording medium 81. Alternatively, the software may be stored in advance in the ROM 72, storage section 79, or the like.

For example, a functional configuration as shown in FIG. 5 is constructed in the CPU 71 of the information processing device 2 using such software (application program).
FIG. 5 shows a UI (user interface) control section 31, a communication control section 32, a text acquisition section 33, a data management section 34, and an upload processing section 35 as functions provided in the information processing device 2.

The UI control unit 31 performs user interface processing such as presenting the image file PF and audio file AF transferred from the imaging device 1 to the user, and accepting user operations for setting and editing various information.
User interface processing includes processing that provides an operation input environment to the user by outputting display output and audio output to the user, processing that performs display output and audio output to present various information to the user, There are processes for detecting user operations, and processes for detecting/estimating user intentions.
The UI control unit 31 performs a process of providing an operation input environment to the user by outputting display output, audio output, etc. to the user, for example.
Alternatively, the UI control unit 31 performs a process of detecting a user's operation, for example.
Alternatively, the UI control unit 31 performs, for example, both the process of providing the user with an operation input environment and the process of detecting the user's operation.
Of course, the UI control unit 31 may perform other user interface processing.

The UI control unit 31 is a UI environment that can turn on/off processing for automatically adding voice memo text, which is a voice memo of an audio file AF, as a part of caption data in metadata added to image data. I will provide a.
The UI control unit 31 also determines whether to automatically upload the image file PF including the image data and metadata to the FTP server 4 after processing the voice memo text into metadata corresponding to the image data. Provides a UI environment that allows you to set In addition, it is also possible to set whether or not to upload the audio file AF in that case.
The UI control unit 31 also provides a UI environment for displaying voice memo text and reproducing audio.

Note that the UI control unit 31 in this embodiment performs user interface processing to enable switching between display and non-display of each item (field) of metadata. It also performs user interface processing to enable operations to change the display name of each item of metadata.

The communication control unit 32 has a function of controlling communication operations by the communication unit 80.
The communication control unit 32 performs a process of causing the communication unit 80 to communicate with the imaging device 1 .

When the audio file AF is transferred from the imaging device 1, the text acquisition unit 33 performs a process of acquiring a voice memo text obtained by converting the voice memo included in the audio file AF into text.
Specifically, the text acquisition unit 33 causes the text conversion engine 83 to execute text conversion processing, and acquires the voice memo text.
Alternatively, the text acquisition unit 33 performs a process of transmitting audio data as a voice memo from the communication unit 80 to the text conversion engine 5 and acquiring the voice memo text returned from the text conversion engine 5.

After receiving the image file PF including the image data transmitted from the imaging device 1 and the audio file AF including the related voice memo, the data management unit 34 receives the voice memo acquired by the text acquisition unit 33 regarding the voice memo included in the audio file AF. This is a function that processes text into metadata corresponding to image data.
The data management unit 34 also performs processing such as storing and editing the image file PF and audio file AF transmitted from the imaging device 1. For example, it performs processing related to editing image data, adding metadata (adding tags), and various operation settings.

The upload processing unit 35 is a function that performs upload processing to the FTP server 4 via the communication unit 80.
In particular, the upload processing unit 35 transfers the image file PF (image data and metadata) to the FTP server 4 after the data management unit 34 processes the voice memo text for the audio file AF into metadata corresponding to the image data. Perform the upload process. The upload processing unit 35 may upload the audio file AF to the FTP server 4 together with the image file PF.

Each of the above functions is provided, for example, by the information processing device 2 installing application software that implements the processing of the present disclosure.
Among the above functions, the information processing device 2 includes the UI control unit 31 to execute various processes for facilitating editing of metadata.

<4. User interface screen>
The UI screen on the display unit 77 of the information processing device 2 will be described below. The following screens are examples of screens that the CPU 71 displays on the display section 77 using the functions of the UI control section 31. In the description, for example, a smartphone is assumed as the information processing device 2, and the display contents on a display formed on the housing thereof are assumed.

FIG. 6 shows the image list screen 50.
The image list screen 50 is a screen on which image data of the image file PF transferred from the imaging device 1 is displayed as a list of thumbnail images.

The image list screen 50 is provided with an image list area 101, and a list of thumbnail images 103 of image data captured from the imaging device 1 is displayed in this image list area 101. Note that you can select to display a list of images, such as all captured images or only protected images. If there are too many images to display on one screen, each image (thumbnail image 103) is displayed by scrolling or turning pages.

Image information 104 is displayed corresponding to each thumbnail image 103. The image information 104 displays, for example, an image data name such as "DSC00000", an icon indicating a protection status, an icon related to FTP upload, and the like.

Also, some of the image data (image files PF) displayed as a list are associated with audio files AF as voice memos. For image data for which a corresponding voice memo exists, a voice memo mark 105 is displayed on the thumbnail image 103 thereof.

A menu button 102 is displayed on the image list screen 50. When the user performs an operation such as tapping the menu button 102, the display transitions to a menu screen 55 to be described later.

FIG. 7 is an example of the menu screen 55. The menu screen 55 is provided with a close button 109 for closing the menu screen 55.

The menu screen 55 includes menu items such as FTP upload preset item 141, IPTC metadata preset item 142, caption template item 143, caption glossary item 144, FTP import history item 145, import item 146, setting item 147, and voice memo auto. A caption addition item 148, a support page item 149, a MAC address confirmation item 150, a data deletion item 151, and an account item 152 are provided. Of course, these are just examples, and various examples of menu item contents are possible. Further, the number of items may be larger or may be hierarchical. If there are many items, each item will be displayed by scrolling or turning the page.

Here, an operation for selecting the setting item 147 will be described as an item related to the processing of this embodiment.
When the setting item 147 is selected, a setting screen 56 shown in FIG. 8 is displayed. The setting screen 56 displays a caption glossary synchronization item 201, an import item 202, a metadata editing setting item 203, and an automatic FTP upload item 204 as setting items. Of course this is just one example. For example, the voice memo automatic captioning item 148 may be displayed as a setting item that can be set on the setting screen 56 shown in FIG. Alternatively, it may be displayed as a setting item that can be set on a screen that changes when a selection operation is performed on the import item 202.
When the user operates the metadata editing setting item 203 on this setting screen 56, a metadata editing setting item screen 66 shown in FIG. 9 is displayed.

The metadata editing setting item screen 66 displays an IPTC metadata embedding method setting item 205 and an IPTC field setting item 206.

The IPTC field setting items 206 allow editing of each item included in the IPTC metadata.
When the IPTC field setting item 206 is selected, an IPTC field setting screen 67 shown in FIG. 10 is displayed.

As shown in FIG. 10, on the IPTC field setting screen 67, each item included in the IPTC metadata is displayed, and controls and display sections for each item are arranged.
Specifically, an item name display section 220, an on/off setting display section 221, and an edit button 222 are provided for each item.

As shown in FIG. 10, the item name display section 220 displays field names that are predetermined for each item of IPTC metadata as specifications. Examples include "Caption/Description" and "Description Writers."

Note that "Caption/Description" is the field name of the aforementioned "Description/Caption" item. Further, "Description Writers" is the field name of the "Description Writer" item mentioned above.

The on/off setting display section 221 exclusively displays icon images that can determine whether the display of each item is on or off. That is, only one of the on-state icon image and the off-state icon image is displayed for each item.
The on/off setting display section 221 also functions as an operator that can change the display on and off states.

Here, the on state and off state of display will be explained.
The display on state is a state that is set for an item that needs to be edited on an IPTC metadata editing screen, which will be described later, and is a state that indicates that the item is displayed on the editing screen.
The display off state is a state that is set for an item that is not edited on an IPTC metadata editing screen, which will be described later, and is a state that indicates that the item is not displayed on the editing screen.
That is, the user can display only the items to be edited on the editing screen by switching the display on and off for each item.
Note that the operation may be performed so that only items that are definitely removed from the editing target are turned off.

The edit button 222 is provided as an operator for editing the display name of each item of IPTC metadata.
Here, field names and display names for each item of IPTC metadata will be explained.
As described above, the field name is determined by the specifications and cannot be changed. However, field names may be difficult for users to understand.
Therefore, in the present embodiment, each item can be displayed with any name set by the user on the program (application) running on the information processing device 2. That is, the "display name" is any name that can be set by the user.

The state shown in FIG. 10 shows a state in which each item is indicated by a field name and a display name has not been set.
Note that among the items in the IPTC metadata, the item name display section 220 in the "Description/Caption" item will be focused on as the item name display section 220A. Also, focus on the edit button 222 in the "Description/Caption" item as the edit button 222A.
Furthermore, the on/off setting display section 221 in the "Headline" item is referred to as an on/off setting display section 221B, and the on/off setting display section 221 in the "keyword" item is focused on as an on/off setting display section 221C.

When the user operates the edit button 222A in the "Description/Caption" item, a field display name change dialog 68 shown in FIG. 11 is displayed in a superimposed manner.
The field display name change dialog 68 displays a default application item 223, a display name input field 224, a change item 225, and a cancel item 226.
In the default application item 223, the default display name, that is, in this example, the character string "Caption/Description" which is the field name is applied to the display name.

The display name input field 224 is a text box for inputting an arbitrary display name. For example, by tapping the display name input field 224, the software keyboard 131 is displayed below the field display name change dialog 68. The user can input a character string into the display name input field 224 by operating the software keyboard 131.

The change item 225 is an item operated when applying the character string input in the display name input field 224 to the display name of the field. When the change item 225 is operated in the state shown in FIG. 11, the display name of the "Description/Caption" item is set to "Description."

The cancel item 226 is an item that is operated when canceling the change of the display name. When the cancel item 226 is operated, the character string input in the display name input field 224 is ignored and the field display name is changed. The state returns to the state before the dialog 68 was displayed.

When the default application item 223, change item 225, or cancel item 226 is operated, the superimposed display of the field display name change dialog 68 ends, and the IPTC field setting screen 67 is displayed.
FIG. 12 is an example of the IPTC field setting screen 67, and shows a case where the change item 225 is operated with "description" entered in the display name input field 224 in FIG. 11.

In the item name display section 220A of FIG. 12, the display name is changed to "Description" and the field name is shown to be "Caption/Description." In this way, the user can change the display name of each item to a character string that is easy to understand. However, since changing the field name is not permitted, the field name remains "Caption/Description".
By displaying both the display name and the field name in the item name display section 220A, it becomes easier to find the item whose display is to be turned on or off, and the relationship with the original name (i.e. field name) is made clear. Therefore, switching errors can be prevented.

The case where the on/off setting display section 221 is operated on the IPTC field setting screen 67 shown in FIG. 12 will be described.
Specifically, when a tap operation is performed on the on/off setting display area 221B in the "Headline" item in FIG. 12 and a tap operation on the on/off setting display area 221C in the "Keyword" item, as shown in FIG. An IPTC field setting screen 67 is displayed.
That is, the "Headline" item is changed from an off state to an on state, and the "keyword" item is changed from an on state to an off state.

Returning to the description of the image list screen 50 shown in FIG. 6.
The user can perform an image selection operation on the image list screen 50. For example, the user can select specific image data by tapping the thumbnail image 103 or the like.
For example, by selecting specific image data, a caption editing screen 52 as shown in FIG. 14 is displayed.

The caption editing screen 52 is provided with an image field 130, a caption field 132, and a voice memo field 133. Further, at the bottom of the caption editing screen 52, an IPTC metadata editing button 240, an FTP upload button 241, a caption editing button 242, and a share button 243 are arranged.

In the image column 130, a thumbnail image 103 and a forward button 107 are displayed. The displayed thumbnail image 103 is a thumbnail image of image data selected by the user. This makes it clear that the displayed caption editing screen 52 is a screen for editing the caption for the image data represented by the thumbnail image 103.

By operating the forward button 107, the user can switch the image list screen 50 to a state in which image data arranged one after the other is selected. For example, by operating the forward button 107, the thumbnail images 103 can be switched on the caption editing screen 52. Then, the caption editing screen 52 becomes a screen for editing the caption for the image data represented by the new thumbnail image 103.

Further, the user can enlarge the image indicated by the thumbnail image 103 by, for example, pinching or tapping.

In the caption field 132 of the caption editing screen 52 in FIG. 14, the caption data described in the "Description/Caption" field in the above IPTC metadata is displayed, and the caption data can be edited.

The caption editing screen 52 is provided with a template button 138 and a voice input button 139.
By operating the template button 138, the user can call up a caption template and display it in the caption field 132.
Furthermore, by operating the voice input button 139, the user can input caption data by voice. In this case, the input voice may be converted into text in the same manner as the voice memo is converted into text.

A voice memo text area 134 is provided in the voice memo field 133 of the caption editing screen 52, and the voice memo text is displayed. That is, the voice memo transferred as audio data from the imaging device 1 is converted into text and displayed in the voice memo field 133. This allows the user to check the contents of the voice memo on the caption editing screen 52.
In addition, a play button 135, a copy button 136, and a delete button 137 are displayed in the voice memo field 133, and operations regarding the voice memo can be performed.

When the user operates the playback button 135, audio playback of the voice memo is started.
For example, during voice playback, instead of displaying the voice memo text area 134, playback button 135, copy button 136, and delete button 137, a seek bar, current time, total playback length, stop button, pause button, etc. are displayed.
Furthermore, when the user operates the delete button 137, the voice memo is deleted.

When the user operates the copy button 136 in the voice memo field 133, the voice memo text displayed in the voice memo text area 134 is copied to the clipboard area on the system.

Note that the voice memo text area 134 may be automatically inserted into the caption field 132 by setting automatic captioning, which will be described later.
For example, FIG. 15 shows an example in which text data as voice memo text is added as caption data with the caption editing screen 52 open. In this case, "Entitled to Base" is the text that was previously input as caption data, and the text that follows "Tanaka hit the ball in the top of the second inning" is the voice memo text that was automatically inserted. .
In addition, when the voice memo text is automatically inserted into the caption data in this way, if the text of the caption data exists first, it is possible to insert the voice memo text before or after the caption. It will be done. In the case of automatic insertion, it is desirable to add voice memo text after the already input caption in order to make the content of the already input caption easier to understand.

When the IPTC metadata editing button 240 on the caption editing screen 52 is operated, an IPTC metadata editing screen 69, which will be described later, is displayed.
Furthermore, when the FTP upload button 241 is operated, a process such as uploading specific image data displayed on the caption editing screen 52 to the FTP server 4 is executed. Note that when uploading, it is possible to specify image size settings, data compression, etc.

When the caption edit button 242 is operated, a software keyboard 131 as shown in FIG. 11 is displayed, and by operating the software keyboard 131, arbitrary characters can be input. The input character string will be written as caption data in the description/caption column of the IPTC metadata for the image data.

When the share button 243 is operated, the specific image data being displayed on the caption editing screen 52 is uploaded to a server device or the like so that it can be viewed by others via an SNS (Social Networking Service) or the like. Note that depending on the public settings, only you may be able to view the page.

Here, an example of the IPTC metadata editing screen 69 that is displayed when the IPTC metadata editing button 240 is operated is shown in FIGS. 16 and 17.
FIG. 16 shows an IPTC metadata editing screen 69 according to the setting state of the IPTC field setting screen 67 of FIG. 12. That is, "Description" is set as the display name of the "Description/Caption" item of the IPTC metadata, the "Headline" item is turned off, and the "Keyword" item is turned on.

FIG. 17 shows an IPTC metadata editing screen 69 according to the setting state of the IPTC field setting screen 67 of FIG. 13. That is, "Description" is displayed as the display name of the "Description/Caption" item of the IPTC metadata, the "Headline" item is turned on, and the "Keyword" item is turned off.

In this way, the display mode of the IPTC metadata editing screen 69 differs depending on the setting state of the IPTC field setting screen 67. That is, by setting only the items to be edited in the ON state on the IPTC field setting screen 67, the user can make the display on the IPTC metadata editing screen 69 easier to see and improve the efficiency of editing work.
Furthermore, by allowing the user to change the display name for each item to a character string of his or her choice, it is possible to prevent the user from selecting the wrong item for editing. Therefore, it is possible to input appropriate text information for each item.
Furthermore, if a display name is set for each item, the field name may be displayed in addition to the display name. This makes the correspondence between the display name and the IPTC metadata item clear, and it is possible to prevent editing mistakes.

When the user performs a selection operation for any item on the IPTC field setting screen 67 shown in FIGS. 16 and 17, a screen for editing the metadata of the selected item is displayed. On the editing screen, characters can be entered using the software keyboard 131 or using voice, and metadata editing work can be performed.
The edited metadata is added to the image data at the timing when the editing for each individual item is completed and the screen transitions to the IPTC metadata editing screen 69 shown in FIGS. 16 and 17. Alternatively, after completing the editing for each individual item and returning to the IPTC metadata editing screen 69, the edited metadata is added to the image data at the timing when an operation to transition from the metadata editing screen 69 to another screen is received. It is also possible to do so.
Note that in the caption editing screen 52 shown in FIG. 14, the caption data input on the caption editing screen 52 is added to the image data at the timing when the return button 106 is operated and the caption editing screen 52 transitions to another screen.
Note that IPTC metadata may be added to image data at other timings. For example, edited metadata may be stored in a storage area used by an application program, and IPTC metadata may be added to image data at the timing when an image to be edited is changed. Alternatively, IPTC metadata may be added to the image data at the timing of transmitting the image file PF.

Next, we will explain the interface screen for settings related to voice memo processing and uploading.
Operations from the setting item 147 and the voice memo automatic captioning item 148 from among the menu items on the menu screen 55 described above will be explained.
The voice memo automatic caption addition item 148 is an item that allows the user to select whether or not to automatically add the voice memo text to caption data when the voice memo is converted into text.

When the user operates the voice memo automatic captioning item 148, the voice memo automatic captioning setting screen 53 of FIG. 18 is displayed. A setting switch 170 is displayed on the voice memo automatic captioning setting screen 53, which allows the user to turn on/off the voice memo automatic captioning function.
When the setting switch 170 is turned on, when a voice memo text is obtained, the voice memo text is automatically inserted into the caption data as shown in FIG. 15.
If the setting switch 170 is turned off, such automatic insertion will not occur.

A return button 106 is provided on the voice memo automatic captioning setting screen 53, and when the return button 106 is operated, the screen returns to the menu screen 55 in FIG. 7. The on/off state of the setting button 170 is enabled in the state when the back button 106 is operated.

Further, when the user operates the setting item 147 on the menu screen 55 of FIG. 7, the setting screen 56 of FIG. 8 is displayed. Displayed on the setting screen are a caption glossary synchronization item 201, an import item 202, a metadata editing setting item 203, and an automatic FTP upload item 204 as setting items.
When the user operates the automatic FTP upload item 204 on this setting screen 56, an automatic upload setting screen 57 shown in FIG. 19 is displayed.

A setting switch 171 is displayed on the automatic upload setting screen 57, which allows the user to turn on/off automatic upload.
The automatic upload function is a function that automatically uploads the image file PF to the set FTP server 4 when the image file PF is transferred from the imaging device 1.
FIG. 19 shows a case where the setting switch 171 is in an off state.
When the user turns on the setting switch 171, a display for setting automatic upload is displayed as shown in FIG. 20. That is, an upload destination display column 175 is displayed, and a setting switch 172 regarding voice memo attachment and a setting switch 173 regarding JPEG image quality are also displayed.

In FIG. 20, the upload destination display column 175 indicates that no upload destination has been specified yet. When the user performs an operation to specify an upload destination, the upload destination specified by, for example, the name "XYZ" given by the user during FTP settings is displayed as shown in FIG.

The setting switch 172 regarding voice memo attachment allows the user to set whether or not to upload the audio file AF as a voice memo together with the image file PF during automatic upload. For example, when the setting switch 172 is turned on as shown in FIG. 21, the audio file AF is also uploaded when automatic upload processing is performed.

A setting switch 173 related to JPEG image quality allows the user to set the compression rate and image size of image data to be uploaded.
When the setting switch 173 is turned on, a compression rate setting bar 176, a long side pixel setting section 177, and a setting switch 174 are displayed as shown in FIG.
The user can specify the compression rate by operating the compression rate setting bar 176. Further, the number of pixels on the long side can be set using the setting switch 174.

The above user operation on the automatic upload setting screen 57 is validated by operating the return button 106 and returning to the setting screen 56 in FIG. 8.
Note that when the return button 106 is operated on the setting screen 56, the screen returns to the menu screen of FIG. 7.

<5. Example of processing when importing images>
Among the various processes executed by the information processing device 2, an example of the process when capturing the image file PF from the imaging device 1 will be described first with reference to FIGS. 22, 23, and 24.
Note that each process example described below is a process executed by the CPU 71 using the functions described in FIG. 5 that are realized by application software.

22 and 23 show a series of flowcharts separately, and "C1" indicates the connection relationship.
In step S101 of FIG. 22, the CPU 71 performs a process of importing the image file PF from the imaging device 1.
The process of importing the image file PF from the imaging device 1 is performed, for example, by communication between the information processing device 2 and the imaging device 1. For example, the user performs a predetermined operation on the information processing device 2 side or the imaging device 1 side. By doing so, the transfer of the image file PF is started.
The imaging device 1 performs processing to transfer the image file PF selected as a transfer target to the information processing device 2 via FTP communication. Further, in this case, if there is an audio file AF having a voice memo associated with the image file PF, that audio file AF is also transferred to the information processing device 2.
On the information processing device 2 side, the CPU 71 performs a process of importing the image file PF and audio file AF that are sequentially transferred as the process of step S101.

Here, there is a rule between the imaging device 1 and the information processing device 2 that if there is an audio file AF associated with the image file PF, the audio file AF is transmitted first, and then the image file PF is transmitted. Decide.
Thereby, when the CPU 71 receives one audio file AF, it can determine that the audio file AF is associated with the next received image file PF. In step S102, the CPU 71 performs a process of linking and managing the received audio file AF with the received image file PF in accordance with such rules.
In addition to this, it is also conceivable that, for example, the metadata of the image file PF includes information that specifies the associated audio file AF. In step S102, the received audio file AF may be linked and managed with the received image file PF with reference to metadata.

When the import of one or more image files PF is completed, the CPU 71 proceeds from step S103 to step S110, and determines whether or not any of the imported image files PF is associated with an audio file AF.
If all the imported files are only image files PF and no audio file AF exists, the CPU 71 proceeds from step S110 to step S120 in FIG. 23.

On the other hand, if there is at least one image file PF associated with the audio file AF, the CPU 71 proceeds from step S110 in FIG. 22 to step S111.
In step S111, the CPU 71 selects one of the one or more image files PF associated with the audio file AF from among the currently imported image files PF as a processing target.

Then, in step S112, the CPU 71 performs text conversion processing on the voice memo of the audio file AF associated with the image file PF targeted for processing.

In step S113, the CPU 71 determines whether text conversion has been performed normally.
If the text conversion is successfully performed and the voice memo text is obtained, the CPU 71 proceeds from step S113 to step S114, and stores the voice memo text in, for example, the storage unit 79 as a voice memo text corresponding to the image file PF to be processed. I do.

In step S115, the CPU 71 checks whether the automatic captioning function is turned on. The automatic captioning function is a function that the user can arbitrarily turn on or off using the voice memo automatic captioning setting screen 53 shown in FIG.
If the automatic captioning function is not on, the CPU 71 proceeds to step S117.

If the automatic captioning function is on, the CPU 71 proceeds to step S116 and performs processing to insert the voice memo text into the caption data. That is, processing is performed to write the voice memo text into the description/caption field in the IPTC metadata. If caption data has already been written in the description/caption field, the CPU 71 subsequently writes the voice memo text. Note that the description/caption field may be overwritten with voice memo text. In that case, the user may be asked to confirm overwriting, or the processing burden may be reduced without confirmation. Furthermore, the insertion position of the voice memo text and whether or not it can be overwritten may be selectable, or an editing screen may be presented to the user.
After performing such automatic captioning processing, the CPU 71 proceeds to step S117.

If the text conversion in step S112 is not performed normally, for example, if the voice memo text cannot be obtained due to a processing error, communication error, etc., the CPU 71 proceeds from step S113 to step S117. In this case, by not particularly retrying the text conversion process, it is possible to avoid prolonging the transfer process. This is because there are other opportunities to convert into text, as described below. However, of course, the text conversion process may be retried a predetermined number of times, or the user may be allowed to select whether or not to proceed with the process without converting the text into text.

If the process proceeds to step S117 after performing the above-described text conversion processing and voice memo automatic captioning processing for a certain image file PF, or if the process proceeds to step S117 without normal completion of text conversion, the CPU 71 It is checked whether there are any remaining image files PF to be subjected to the same processing, and if there are, the process returns to step S111, one of which is targeted for processing, and the same processing as described above is performed.
By executing the text conversion process of step S112 at least once for all image files PF associated with the audio file AF, it is determined in step S117 that all text conversion has been completed, and the process proceeds to step S120 in FIG. 23.

In step S120, the CPU 71 checks whether the automatic upload function is turned on. The automatic upload function is a function that the user can arbitrarily turn on/off on the automatic upload setting screen 57 shown in FIGS. 19, 20, and 21.
If the automatic upload function is not on, the CPU 71 finishes the series of processes at the time of image capture from step S120.

When the automatic upload function is on, the CPU 71 proceeds from step S120 to step S121, and performs upload file preparation processing.
This upload file preparation process is shown in detail in FIG.

In step S141 of FIG. 24, the CPU 71 specifies one of the image files PF to be uploaded. This means that one of the image files PF transferred from the imaging device 1 this time is targeted for preparation processing.
If one image file PF is to be processed, the CPU 71 checks in step S142 whether an image size has been specified. This means that the user confirms the contents set by the long side pixel setting section 177 and setting switch 174 on the automatic upload setting screen 57 shown in FIG. If the image size has been specified, the CPU 71 performs a process of converting the number of pixels to reflect the specification in step S143.

In step S144, the CPU 71 checks whether a compression ratio has been specified. This means checking the specified state of the compression rate using the compression rate setting bar 176 on the automatic upload setting screen 57 in FIG. 21. When the compression rate is specified and compression is set to be performed, the CPU 71 performs compression processing using the specified compression rate in step S145.

In step S146, the CPU 71 checks whether the audio file AF is attached. That is, the setting of whether or not to upload the user's voice memo is confirmed by turning on/off the setting switch 172 in FIG. 21.
If uploading the audio file AF is selected, the CPU 71 proceeds to step S147 and checks whether there is an audio file AF associated with the image file PF currently being processed. If the related audio file AF exists, the CPU 71 proceeds to step S149 and sets the current processing target image file PF (image data and metadata) and audio file AF as files to be uploaded.

If it is confirmed in step S146 that uploading the audio file AF is not selected as a setting by the user, or if there is no audio file AF associated with the image file PF currently being processed in step S147, The CPU 71 proceeds to step S148 and sets the current image file PF (image data and metadata) to be processed as the file to be uploaded.

In step S150, it is checked whether the above preparation process has been completed for all image files PF imported from the imaging device 1 this time. If there are remaining image files PF, the CPU 71 returns to step S141, specifies one of the remaining image files PF as a processing target, and performs the same processing.
If it is determined in step S150 that the above preparation process has been completed for all image files PF imported from the imaging device 1 this time, the CPU 71 finishes the upload file preparation process shown in FIG. 24.
That is, the CPU 71 proceeds to step S122 in FIG. 23.

In step S122, the CPU 71 performs FTP connection and login processing to the FTP server 4 specified by the automatic upload settings.
When the login is completed, the CPU 71 advances from step S123 to step S130 and executes FTP upload processing. That is, a process is performed in which image files PF and audio files AF that are set to be uploaded in the upload file preparation process are sequentially transmitted via FTP.

Note that if communication with the FTP server 4 is not possible, or if communication is possible but login is not possible, the CPU 71 determines an error and proceeds from step S124 to step S125 to perform predetermined error processing. For example, notify users of errors in automatic upload processing. Then, the series of processes at the time of importing the image file PF from the imaging device 1 is completed.

If the FTP upload in step S130 is successfully completed, the CPU 71 proceeds from step S131 to step S133, notifies the user of completion, and ends the series of processing. In this case, when the image file PF is taken in from the imaging device 1, uploading to the FTP server 4 is automatically performed.
Furthermore, if the voice memo automatic captioning function is on, in the uploaded image file PF, the voice memo text, which is the voice memo of the associated audio file AF converted into text, is added to the IPTC metadata. There will be.

If an error occurs in the upload process to the FTP server 4 and the process cannot be completed normally, the CPU 71 proceeds from step S132 to step S134 and performs predetermined error processing. For example, notify users of errors in automatic upload processing. Then, the series of processes at the time of importing the image file PF from the imaging device 1 is completed.

<6. Processing example including text conversion and caption editing>
Next, a processing example when the image list screen 50 of FIG. 6 is displayed transitions to the caption editing screen 52 of FIG. 14 etc. will be described with reference to FIGS. 25, 26, and 27.
Note that FIGS. 25, 26, and 27 show a series of flowcharts separately, and the connection relationships are shown by "C2", "C3", "C4", "C5", and "C6".

Step S201 in FIG. 25 assumes that the CPU 71 is displaying the image list screen 50.
As described above, when the user performs an operation to designate a certain image on the image list screen 50, the caption editing screen 52 for that image data is displayed. At this time, if the voice memo has not yet been converted into text for the designated image data (image file PF), the voice memo is converted into text at this opportunity.

When the CPU 71 detects an image designation operation by the user on the image list screen 50, the CPU 71 proceeds from step S202 to step S203.
In step S203, the CPU 71 checks whether there is an audio file AF associated with the specified image data (image file PF).
If the associated audio file AF does not exist, the process advances to step S220, and the CPU 71 displays the caption editing screen 52 for the designated image data. In this case, since no voice memo exists, the voice memo field 133 does not need to be displayed on the caption editing screen 52.

If the associated audio file AF exists, the CPU 71 proceeds to step S204 and checks whether the voice memo has already been converted into text and the voice memo text is stored. If it has already been stored, the process advances to step S220, and the CPU 71 displays the caption editing screen 52 for the specified image data. In this case, a voice memo field 133 displaying a voice memo text area 134 will be displayed as shown in FIG. Further, if the automatic captioning function is turned on and the voice memo text is inserted into the caption data, the voice memo text is also added to the caption data in the caption field 132, as shown in FIG. 19.

If it is determined that the voice memo exists but has not been converted into text, the CPU 71 proceeds to step S205 and performs text conversion processing on the voice memo of the audio file AF associated with the designated image file PF. For example, audio data as a voice memo is sent to the text conversion engine 5, and text data is received. Alternatively, if the information processing device 2 is equipped with the text conversion engine 83, the text conversion engine 83 may perform the text conversion process.

If the text conversion is successfully performed and the voice memo text is obtained, the CPU 71 proceeds from step S206 to step S207, and stores the voice memo text in, for example, the storage unit 79 as a voice memo text corresponding to the image file PF to be processed. I do.

In step S208, the CPU 71 checks whether the automatic captioning function is turned on. The automatic captioning function is a function that the user can arbitrarily turn on or off using the voice memo automatic captioning setting screen 53 shown in FIG.
If the automatic captioning function is not on, the CPU 71 proceeds to step S220. In this case, a voice memo field 133 is displayed on the caption editing screen 52, and in the caption field 132, caption data that has been input up to that point is displayed.

On the other hand, if the automatic captioning function is on, the CPU 71 proceeds to step S209 and performs a process of inserting the voice memo text into the caption data. That is, processing is performed to write the voice memo text into the description/caption field in the IPTC metadata. As described above, if caption data has already been written in the description/caption field, the CPU 71 subsequently writes the voice memo text.
The CPU 71 performs such automatic captioning processing and then proceeds to step S220. In this case, the caption editing screen 52 is in a state where the voice memo text is also added to the caption data in the caption field 132, as shown in FIG.

If the text conversion in step S205 is not performed normally, the CPU 71 proceeds from step S206 to step S220. In this case, the voice memo field 133 may not be displayed on the caption editing screen 52 because the voice memo has not been converted into text. However, it is conceivable that the voice memo mark 105 clearly indicates the existence of the voice memo to the user.

With the caption editing screen 52 displayed, the process of the CPU 71 proceeds to step S221 in FIG. 26. The CPU 71 monitors various user operations on the caption editing screen 52. That is, each operation is monitored in a loop of steps S221, S222, S223, S224, S225, and steps S226, S227, S228, and S229 in FIG.

In step S221 of FIG. 26, the CPU 71 monitors the screen transition by the return button 106, that is, the operation of transition to the image list screen 50. If this operation is detected, the CPU 71 performs a caption saving process in step S240, and returns to step S201 in FIG. 25. That is, the caption data displayed in the caption field 132 at that time is saved as data in the description/caption field in the IPTC metadata.

In step S222, the CPU 71 monitors operations related to caption input. In response to operations related to caption input, such as character input, voice input, or template request, the CPU 71 performs input response processing in step S241.
In the input handling process, for example, display processing of the software keyboard 131 is performed. In addition, character input and deletion processing, or caret position change processing, etc., are performed in response to operations on the software keyboard 131. Further, copy processing, paste processing, etc. are also performed in accordance with the copy operation and paste operation.

The CPU 71 monitors the user's operation of the playback button 135 in step S223. If the CPU 71 detects the operation of the playback button 135, the CPU 71 proceeds to step S242 and performs control to start audio playback.

The CPU 71 monitors the user's operation of the pause button in step S224. If the operation of the pause button is detected, the CPU 71 proceeds to step S243 and performs control to temporarily stop audio reproduction. Although not shown, in this case, the pause button is switched to display the play button 135.

The CPU 71 monitors the user's operation of the stop button in step S225. If the operation of the stop button is detected, the CPU 71 proceeds to step S244 and performs control to stop audio reproduction. In this case, the display in the voice memo field 133 is returned to the state shown in FIG. 14.

The CPU 71 monitors the user's operation of the copy button 136 in step S226 of FIG. If the operation of the copy button 136 is detected, the CPU 71 proceeds to step S245 and performs a process of copying the voice memo text to the clipboard. Then, in step S246, the CPU 71 displays a message regarding copying for a predetermined period of time or until the next user operation is detected.
Note that the voice memo text copied to the clipboard will be pasted as caption data in step S241 when the CPU 71 detects a paste operation in step S222.

The CPU 71 monitors the user's operation of the delete button 137 in step S227. If the operation of the delete button 137 is detected, the CPU 71 proceeds to step S247 and displays a confirmation dialog for the user to confirm whether or not to delete. When the user performs an operation indicating permission for deletion, the CPU 71 proceeds from step S248 to step S250, and performs deletion processing regarding the voice memo.
On the other hand, when the user performs an operation indicating that deletion is not permitted, the CPU 71 proceeds from step S248 to step S249, and detects whether or not there is an operation to cancel deletion. If the user indicates an intention to cancel, the CPU 71 does not execute the deletion process and returns to monitoring the user operation by returning from step S249 to step S221. If the intention to cancel is not expressed, the CPU 71 executes the process of step S247 again. That is, unless an operation to permit deletion or an operation to cancel is performed, the CPU 71 repeats each process of steps S247, S248, and S249.

The CPU 71 monitors the image enlargement operation in step S228. For example, when an image enlargement operation is performed such as a pinch operation or a double tap on the thumbnail image 103, an individual image screen is displayed in which each individual image is enlarged and displayed.
Although not shown in the flowchart, after transitioning to the individual image screen, a different image is selected according to a predetermined operation, the screen returns to the image list screen 50, or the screen transitions to the caption editing screen 52. .

In step S229, the CPU 71 monitors other image selection operations. That is, the operation of the forward button 107 on the caption editing screen 52 is monitored. When detecting the operation of the forward button 107, the CPU 71 proceeds to step S203 in FIG. 25. In other words, assuming that a previous or subsequent image is newly specified, the processes from step S203 to step S209 for the specified image data are performed in the same manner as described above, and in step S220, the caption editing screen 52 for that image data is displayed. I do.

In step S230, the CPU 71 monitors whether the IPTC metadata edit button 240 has been operated. If an operation on the IPTC metadata editing button 240 is detected, the CPU 71 performs processing to display the IPTC metadata editing screen 69 (see FIGS. 16 and 17) in step S231. When displaying the IPTC metadata editing screen 69, for example, the display on/off settings for each item of IPTC metadata are acquired from the storage unit 79, and the items are displayed accordingly. That is, processing is performed so that some items are hidden.

When one piece of image data is specified as described above, text conversion processing or automatic captioning function processing may be performed. In addition, on the caption editing screen 52, it is possible to display, reproduce, delete, copy, and paste voice memo text into caption data, and input captions manually or by voice.

<7. Example of processing when uploading>
FIG. 28 shows the process when uploading to the FTP server 4. For example, the upload process shown in FIG. 28 is started when the user specifies the image file PF to be uploaded and the FTP server 4 as the upload destination and issues an instruction to execute the upload.

The CPU 71 performs upload file preparation processing in step S301. This is the same process as in FIG. 24 described above.
In step S322, the CPU 71 performs FTP connection and login processing to the FTP server 4 as the upload destination specified by the user.
When the login is completed, the CPU 71 advances from step S323 to step S330 and executes FTP upload processing. That is, a process is performed in which image files PF and audio files AF that are set to be uploaded in the upload file preparation process are sequentially transmitted via FTP.

Note that if communication with the FTP server 4 is not possible, or if communication is possible but login is not possible, the CPU 71 determines an error and proceeds from step S324 to step S325 to perform predetermined error processing. For example, notify the user of an error in the upload process. Then, the upload process ends with an error.

If the FTP upload in step S330 is successfully completed, the CPU 71 proceeds from step S331 to step S333, notifies the user of completion, and finishes the upload process.
In the uploaded image file PF, it is assumed that the voice memo text is reflected as caption data in the IPTC metadata by the automatic captioning function or by the user's copy/paste operation.
It is also assumed that an audio file AF may also be uploaded according to the user's settings.

If an error occurs in the upload process to the FTP server 4 and the process cannot be completed normally, the CPU 71 proceeds from step S332 to step S334 and performs predetermined error processing. For example, notify the user of an error in the upload process. Then, the upload process ends with an error.

<8. Processing related to IPTC field setting screen>
An example of a process executed by the information processing device 2 in a state where the IPTC field setting screen 67 shown in FIG. 10 etc. is displayed will be described with reference to FIG. 29.

The CPU 71 monitors the operation on the on/off setting display section 221 in step S401.
If it is determined that the on/off setting display section 221 has been operated, the CPU 71 performs switching processing in step S402. In the switching process, a process is performed to switch between an on state and an off state for an item to be operated among each item of IPTC metadata.
After completing the switching process, the CPU 71 returns to the process of step S401 again.

In step S403, the CPU 71 monitors the operation on the edit button 222.
If it is determined that the edit button 222 has been operated, the CPU 71 displays a field display name change dialog 68 shown in FIG. 11 in step S404. This makes it possible to change the field display name to any character string.

After completing the process in step S404, the CPU 71 detects various operations performed by the user while the field display name change dialog 68 is displayed, and performs corresponding processes. For example, as described above, the process of detecting a display name input operation and displaying the input characters in the display name input field 224, or detecting the operation of determining an input character string as a display name and storing the display name of the field. Perform processing, etc.
Upon ending the display of the field display name change dialog 68, the CPU 71 returns to the process of step S401 again.

The CPU 71 monitors screen transition operations such as operations on the return button 106 in step S405.
For example, if it is determined that the return button 106 has been operated, the CPU 71 performs processing to display the metadata editing setting item screen 66 in step S406. Note that when transitioning from the IPTC field setting screen 67 to another screen, the CPU 71 performs a process of storing various setting situations reflected on the screen in the storage unit 79. As a result, display/non-display settings for each item of IPTC metadata, display name for each field, etc. are memorized, and when displaying the IPTC metadata editing screen 69 shown in FIG. 17, the optimal display for the user is created. It can be carried out.

The CPU 71 repeats the series of processes shown in FIG. 29 while the IPTC field setting screen 67 is displayed.

<9. Summary and modifications>
The above embodiment provides the following effects.
The information processing device 2 of the embodiment includes a user interface control unit (UI control part 31).
This makes it possible to display only those fields included in the metadata that the user wants to edit.
Therefore, it is possible to provide an environment in which it is easy for the user to search for fields to be edited, and to improve the efficiency of metadata editing work.

As explained in the processing example related to the IPTC field setting screen 67 in FIG. Processing to control display availability may also be performed.
Thereby, the result of the user's switching operation is reflected on the display.
Therefore, it is possible to provide a display that is convenient for users who edit metadata.

As described in the processing example related to the IPTC field setting screen in FIG. 29, the storage unit 79 may be provided to store settings for display permission/prohibition.
Thereby, the result of the user's switching operation is saved and used for display.
Therefore, it is possible to actually provide an environment in which only what the user wants to edit is displayed in response to the user's switching operation performed for each item of metadata.

As explained in the IPTC metadata editing screen 69 in FIGS. 16 and 17, the user interface control unit (UI control unit 31) selects the field (item) of the displayed metadata (IPTC metadata, etc.) based on whether or not it can be displayed. ) may perform a process of detecting an operation of selecting a field (item) to be edited.
This allows the user to edit the item to be edited. In particular, since the display can be switched on and off so that only the items to be edited are displayed, it is easier to select the items to be edited.
Therefore, it is possible to improve the efficiency of editing work.

As explained in connection with the IPTC field setting screen 67 in FIG. 10, the user interface control unit (UI control unit 31) performs user interface processing to enable editing of display names for each field (IPTC metadata items, etc.). Good too.
This makes it possible to perform an operation to give a different display name when the field name of metadata is difficult to understand.
Therefore, it is possible to provide an environment in which the user can easily search for fields to be edited, and to improve the efficiency of metadata editing work.

As described for the item name display section 220A in FIG. 12, the user interface control section (UI control section 31) may perform user interface processing so that field names cannot be edited.
This prevents, for example, a field name determined by the specifications from being changed and image file classification processing based on metadata not functioning properly.

As explained in the IPTC field setting screen 67 in FIGS. 16 and 17, the user interface control unit (UI control unit 31) performs processing to detect an operation to edit text information as metadata (IPTC metadata, etc.). You may go.
This allows the user to edit the text information of the metadata.
Therefore, it is possible to improve the efficiency of editing work.

As described about the information processing device 2 using FIG. 1, the information processing device 2 is preferably a mobile terminal device.
Thereby, images captured by the imaging device 1 can be transferred to the mobile terminal device serving as the information processing device 2 on the spot, and metadata can be edited quickly. In particular, by displaying only the metadata items to be edited, the editing work can be made more efficient.

As explained using FIG. 3, the metadata may be IPTC metadata. By using metadata formatted by an organization, etc., the above-mentioned technology can be widely applied.

In the explanation of the flowchart in FIG. 25, an example was described in which the text conversion process is executed when the voice memo has not been converted to text when transitioning from the image list screen 50 to the caption editing screen 52. In order to do this, the text conversion process may be executed at other timings.
For example, an example has been described in which text conversion processing is executed when an image is captured from an imaging device in step S112 of FIG. 22. At this time, if it is determined in step S113 that the voice memo has not been successfully completed, a dialog is displayed to notify that the conversion of the voice memo to text has failed, and a retry button for retrying the conversion to text is placed on the dialog screen. good. That is, the text conversion process may be executed again by the user operating the retry button.

Furthermore, in the above description, it was stated that IPTC metadata may be added to image data at a timing other than screen transition, but in that case, it is possible to select the image file PF to which IPTC metadata is added , IPTC metadata may be added to an arbitrary number of image files PF.
For example, by operating the IPTC metadata preset item 142 on the menu screen 55 in FIG. 7, preset data of any IPTC metadata can be created. Furthermore, by performing a predetermined operation after selecting an arbitrary number of images on the image list screen 50, the user can add preset data of IPTC metadata to a plurality of selected image files PF at once. It is considered possible.

Preset data of IPTC metadata can be created not only by creating new data, but also by editing existing preset data to create new preset data. Further, the user may be able to perform operations such as displaying a list of preset data, deleting some of the preset data, and editing already created preset data.
It may be possible to add preset data of IPTC metadata to one or more image files PF by selecting arbitrary preset data from a list of preset data and then selecting an image file PF to be added.

IPTC metadata may not only be generated using the above-mentioned application program, but may also be obtainable from another information processing device. For example, preset data of IPTC metadata generated by another information processing device is imported into the local storage of the information processing device 2 as an xmp (Extensible Metadata Platform) file, and from the local storage, a storage area (application program It is possible to store it as preset data in the DB used by The preset data stored in the storage area is handled in the same way as preset data generated by the information processing device 2 using an application program.

When storing in a storage area managed by an application program, among the items of IPTC metadata, only the items to be processed by the application program are stored as preset data. For example, if an item newly added due to a revision of IPTC metadata specifications cannot be handled by an application program, it may be excluded from processing. In this case, only the items to be processed are stored as preset data in the storage area managed by the application program.
Alternatively, items created by an unknown tool may be excluded from processing.

Note that the preset data of IPTC metadata is stored for each user (each account). For example, preset data of IPTC metadata is managed in association with a user ID. Preset data for each user may be shared so that other users can use it.

In assigning preset data of IPTC metadata, if IPTC metadata has already been assigned to the image file PF to be assigned, the IPTC metadata is overwritten. Note that a process may be performed in which the user is asked whether or not to overwrite the IPTC metadata. Also, for items that can be added among the IPTC metadata items, by writing the preset data that is added this time to the metadata that was written before adding the preset data, both information will be saved in the image file PF. may be given to

The program of the embodiment is a program that causes a CPU, a DSP, etc., or a device including these, to execute each process shown in FIGS. 22 to 29, for example.
That is, the program of the embodiment is a program that causes the information processing apparatus to execute a process of detecting an on/off switching operation for displaying each field of metadata added to an image.
With such a program, the above-described information processing device 2 can be implemented in, for example, a mobile terminal device, a personal computer, or other equipment capable of executing information processing.

A program for realizing such an information processing device 2 can be recorded in advance in an HDD as a recording medium built into a device such as a computer device, or a ROM in a microcomputer having a CPU.
Alternatively, a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, It can be stored (recorded) temporarily or permanently in a removable recording medium such as a memory card. Such a removable recording medium can be provided as so-called package software.
In addition to installing such a program into a personal computer or the like from a removable recording medium, it can also be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

Further, such a program is suitable for widely providing the information processing device 2 of the embodiment. For example, by downloading a program to a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), etc., the smartphone or the like can be used as the information processing device 2 of the present disclosure. It can be made to work.

Note that the effects described in this specification are merely examples and are not limiting, and other effects may also exist.

<10. This technology>

Note that the present technology can also adopt the following configuration.
(1)
An information processing device including a user interface control unit that performs processing to detect an on/off switching operation for displaying each field of metadata added to an image.
(2)
The information processing device according to (1), wherein the user interface control unit performs a process of controlling whether or not the metadata field can be displayed based on the detected switching operation.
(3)
The information processing device according to (2) above, further comprising a storage unit that stores the display enable/disable setting.
(4)
The user interface control unit performs a process of detecting an operation of selecting a field to be edited from among the fields of the metadata displayed based on the display availability, as described in any one of (2) to (3) above. information processing equipment.
(5)
The information processing device according to any one of (1) to (4) above, wherein the user interface control unit performs user interface processing that allows editing of a display name for each field.
(6)
The information processing device according to (5), wherein the user interface control unit performs the user interface processing so that field names cannot be edited.
(7)
The information processing device according to any one of (1) to (6) above, wherein the user interface control unit performs a process of detecting an operation to edit text information as the metadata.
(8)
The information processing device according to any one of (1) to (7) above, which is a mobile terminal device.
(9)
The information processing device according to any one of (1) to (8) above, wherein the metadata is IPTC metadata.
(10)
An information processing method in which an information processing device performs processing to detect on/off switching operations for displaying each field of metadata added to an image.
(11)
A program that causes an information processing device to perform processing to detect on/off switching operations for the display of each field of metadata added to an image.

1 Imaging device 2 Information processing device 4 FTP server 6 Network 31 UI control unit 67 IPTC field setting screen 68 Field display name change dialog 69 IPTC metadata editing screen 71 CPU
79 Storage section 221 On/off setting display section 222 Edit button

Claims (10)

Equipped with a user interface control unit that performs processing to detect on/off switching operations for displaying each field of metadata added to an image ,
The user interface control unit performs user interface processing to change the display name of a field whose stored information content is standardized.
Information processing device. The information processing apparatus according to claim 1 , wherein the user interface control unit performs processing for controlling whether or not the metadata field can be displayed based on the detected switching operation. The information processing apparatus according to claim 2, further comprising a storage unit that stores the display permission/prohibition setting. The information processing apparatus according to claim 2, wherein the user interface control unit performs a process of detecting an operation for selecting a field to be changed from among fields of the displayed metadata based on the display availability. The information processing apparatus according to claim 1 , wherein the user interface control unit performs the user interface processing so that field names cannot be edited. The information processing apparatus according to claim 1, wherein the user interface control unit performs processing to detect an operation to edit text information as the metadata. The information processing device according to claim 1, which is a mobile terminal device. The information processing device according to claim 1, wherein the metadata is IPTC metadata. A process for detecting an on/off switching operation for displaying each field of metadata added to an image ;
A process for changing the display name of a field whose stored information content is standardized;
An information processing method executed by an information processing device. A process for detecting an on/off switching operation for displaying each field of metadata added to an image ;
A process for changing the display name of a field whose stored information content is standardized;
A program that is executed by an information processing device.

Images