JP2023030849A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

To store image data for which image analysis has failed, more efficiently.SOLUTION: Image registration means acquires and registers image data to be analyzed. Image analysis means analyzes the image data acquired from the image registration means, as image data to be analyzed, and extracts metadata. Error image storage necessity determination means acquires information on error image data for which analysis error has occurred in image analysis of the image analysis means, from an image management table generated by the image registration means. Error image storage condition management means acquires the number of pieces of error image data stored already in error image storage means. The error image storage condition management means acquires storage conditions of the error image data, set in advance by a user. The error image storage necessity determination means determines whether to store the error image data, on the basis of the number of pieces of stored error image data and the storage conditions of the error image data.SELECTED DRAWING: Figure 5

Description

The present invention relates to technology for managing image data.

A photo storage service provided via the Internet provides a function of storing image data uploaded by a user in a storage, and viewing and editing the stored image data. Some photo storage services offer the ability to parse uploaded image data and extract metadata. The metadata includes the date and time and location of the image taken to obtain the image data, information on the imaging device that generated the image data, size information indicating the number of pixels in the width direction and height direction of the image data, and the like. mentioned.

Metadata extraction for image data may fail depending on the type and state of the image data, but such image data for which metadata extraction fails should be stored in order to investigate the cause of the failure. back. By identifying the cause of failures in stored image data and modifying the metadata extraction method, failures can be reduced and metadata can be extracted from more image data. Since there is a limit to the amount of image data that can be stored, it is possible to manage image data so that it is possible to store more useful image data by determining whether storage is necessary for each image data. ing.

Japanese Patent Application Laid-Open No. 2002-200003 discloses a technique of setting priority to image data based on a subject appearing in an image, and determining whether or not to store the image data based on the priority. For example, when a certain machine breaks down, a high priority is set for storage of image data obtained by imaging the broken machine.

Japanese Patent Application Laid-Open No. 2019-29001

However, in Japanese Patent Application Laid-Open No. 2002-200012, since the necessity of storage is determined based on the subject appearing in the image, when there is a large amount of image data obtained by imaging the subject to be stored, a plurality of similar images are stored. There is a problem that image data can be stored in a larger amount than necessary. Storing more image data than necessary results in unnecessary costs for storage capacity and image data processing.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to more efficiently store image data for which image analysis has failed.

In order to solve the above-mentioned problems, the present invention provides an information processing apparatus comprising: extracting means for acquiring image data for which image analysis has failed in image analyzing means, and extracting metadata from the image data; and management means for storing the image data from which the metadata has been extracted in a storage device if a predetermined storage condition is satisfied.

According to the present invention, image data for which image analysis has failed can be stored more efficiently.

A diagram showing the configuration of the image archiving device according to the first embodiment. 2 is a diagram showing the hardware configuration of the information processing apparatus according to the first embodiment; FIG. Diagram showing the overall processing flow of the image storage device Diagram showing image management table A diagram showing a processing flow for determining whether or not storage is necessary in the first embodiment Storage condition table in Embodiment 1 A diagram showing a processing flow for determining whether or not storage is necessary using information about an imaging device in the second embodiment. A diagram showing a processing flow for determining whether or not storage is necessary using error details in the second embodiment. Storage condition table in Embodiment 2 A diagram showing the configuration of an image archiving device according to the third embodiment. A diagram showing an interface for setting storage conditions in the third embodiment. A diagram showing a storage condition change notification according to the third embodiment.

Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[Embodiment 1]
FIG. 1 shows a functional block diagram of an image storage device according to this embodiment.

The image storage device 101 in this embodiment has image registration means 102 , image analysis means 103 , and image storage means 104 . The image registration means 102 is used to acquire image data to be analyzed and the information of the image data is registered, and the image analysis means 103 is used to analyze the registered image data. The image data successfully analyzed by the image analysis means 103 is stored in the image storage means 104 .

The image storage device 101 in this embodiment further includes an error image storage necessary/unnecessary determination unit 105 , an error image storage condition management unit 106 , and an error image storage unit 107 . An error image storage necessity determination unit 105 is used to determine whether storage is necessary for image data for which analysis by the image analysis unit 103 has failed. The error image storage necessity determining unit 105 acquires the storage conditions for the error image data, which is the image data whose analysis failed, from the error image storage condition management unit 106, and determines whether storage is necessary. Of the image data for which the analysis by the image analysis means 103 has failed, the image data determined to require storage is stored in the error image storage means 107 .

Note that the image storage unit 104 and the error image storage unit 107 may be provided in the image storage device 101 as shown in FIG. 1, or may be configured to use a function published as a storage service via a network.

FIG. 2 shows a typical hardware configuration of an information processing device capable of realizing the image storage device 101 in this embodiment. As shown in the figure, the information processing device capable of realizing the image storage device 101 includes a CPU (Central Processing Unit) 201 , a ROM (Read Only Memory) 202 and a RAM (Random Access Memory) 203 . Further, this information processing apparatus includes a NET I/F (Network Interface) 204 , a display device 205 , an input device 206 and a bus 207 .

The CPU 201 controls each device connected to the bus 207 . The CPU 201 also executes various processes in the image storage device 101 . The ROM 202 stores an operating system that performs basic control of the computer, operating programs, and the like. A RAM 203 serves as a work memory for the CPU 201 and stores the operation program itself and data necessary for the operation program. A NET I/F 204 controls exchange of information with an external device connected via a network. When each unit of the image storage device 101 is implemented as separate hardware, the NET I/F 204 is used to transmit and receive data.

A display device 205 is for displaying the processing result of the CPU 201, and is configured by a liquid crystal display, an organic EL display, or the like. The input device 206 is for physically accepting input from the user, and is configured by a keyboard, mouse, touch panel, or the like. In the case of a display with a touch panel, the display device 205 and the input device 206 are integrated. Also, it is possible to use the display device 205 and the input device 206 of the external device through the NET I/F 204, in which case the configuration is such that the display device 205 and the input device 206 are not provided.

FIG. 3 shows a flowchart for explaining specific processing when the image storage apparatus 101 in this embodiment determines whether error image data needs to be stored.

In S301, the image registration unit 102 acquires and registers image data to be analyzed. Details of the processing of S301 will be described later. Image data to be analyzed may be a still image or a moving image. Image data may be registered one by one, or a plurality of image data may be registered at once.

In S302, the image analysis unit 103 analyzes the image data acquired from the image registration unit 102 as image data to be analyzed, and extracts metadata. The image data may include an identifier or the like that uniquely identifies the image data. The metadata to be extracted includes the image data format, date and time when the image data was created, size information such as the number of pixels in the width and height directions of the image, and imaging information such as the lens name and shutter speed used when shooting. can do. Any image analysis library can be used to analyze the image data. After the image analysis is performed, information is output as to whether the image analysis was successful.

Note that the items listed as the metadata of the image data are only examples, and other items may be extracted. For example, as metadata of image data, data recorded in image data in Exif (Exchangeable image file format) may be extracted.

In S303, the image analysis means 103 determines whether the image analysis performed in S302 was successful. If the image analysis succeeds, the process proceeds to S304, and if the image analysis fails, the process proceeds to S305.

In S<b>304 , the image analysis unit 103 stores the image data and metadata for which the image analysis has succeeded in the image storage unit 104 .

In S305, the error image storage necessity determining unit 105 determines whether or not it is necessary to store the error image data for which the image analysis unit 103 fails the image analysis. Details of the processing of S305 will be described later with reference to FIGS. If it is determined that the error image data needs to be stored, the process advances to step S306, and if it is determined that the storage is unnecessary, the process ends without storing the error image data.

In S<b>306 , the error image storage necessary/unnecessary determination unit 105 stores in the error image storage unit 107 the error image data determined to require storage among the error image data for which the image analysis has failed. The error image storage unit 107 stores the image data issued at the time of registration of the image data in S301 in association with an identifier that uniquely identifies the image data. Note that when instructing the error image storage unit 107 to store the error image data, it is also possible to set a retention period for the error image data and delete the image data exceeding the retention period. Alternatively, the error image storage means 107 as a whole may set a uniform retention period for image data.

FIG. 4 shows an image management table generated when image data is registered in S301.

When registering image data, the image registration means 102 issues an image ID 401 that uniquely identifies image data to be analyzed, and records the issued image ID 401 in the image management table illustrated in FIG. The image registration means 102 stores the image data in the image storage means 104 in association with the image ID 401 .

In the image management table shown in FIG. 4, in addition to the image ID 401 that is the identification information of the image data, the identification information of the imaging device that generated the image data is recorded. As the identification information of the imaging device, a device ID 402 that uniquely identifies the imaging device and a model ID 403 that identifies the model of the imaging device are recorded. Furthermore, the firmware version 404 of the imaging device and the serial number 405 of the imaging device are recorded. In the example shown in FIG. 4, two image data whose image IDs 401 are "image_001" and "image_002" are image data obtained by imaging with the same model imaging apparatus identified as "model_001". showing. Note that the items of the image management table shown in FIG. 4 are merely an example, and other items related to image data and processing performed on the image data may be recorded. For example, the date and time when the image data was registered may be recorded. Also, the items recorded in the image management table may be divided and recorded in a plurality of tables. For example, items related to image data and items related to imaging devices may be recorded in different tables.

Next, FIG. 5 shows a flowchart for explaining specific processing when the error image storage necessity determination unit 105 in this embodiment determines in S305 whether error image data storage necessity is required. Also, FIG. 6 shows a storage condition table used in this process.

In S<b>501 , the error image storage necessity determination unit 105 acquires information on the error image data in which the analysis error has occurred from the image management table generated by the image registration unit 102 . Based on the image ID 401 of the image data in which the analysis error occurred, the model ID 403 of the imaging device that generated the image data is obtained. Also, the format of the image data is acquired from the extension of the image data in which the analysis error occurred. Note that the format may be specified from the information inside the image data.

In S<b>502 , the error image storage condition management unit 106 acquires the number of error image data already stored in the error image storage unit 107 . The error image storage condition management unit 106 acquires a list of error image data stored in the error image storage unit 107, and acquires the total number of error image data whose model ID 403 and format match among the error image data. . The error image storage means 107 may have an error image management table for managing the number of stored error image data for each combination of model ID 403 and format.

In S503, the error image storage condition management unit 106 acquires an error image data storage condition preset by the user. In this embodiment, the error image storage condition management unit 106 manages the maximum storage number 602 for each combination of the model ID 403 and the format 601 using the storage condition table illustrated in FIG. Based on the information of the model ID 403 and the format 601 acquired in S501, the maximum storage number 602 for each combination of the model ID 403 and the format 601 is acquired. If it is not possible to obtain a storage condition in which the maximum storage number 602 corresponding to the combination of the model ID 403 and the format 601 is preset by the user, a predetermined default value may be applied.

In S504, the error image storage necessity determination unit 105 determines whether storage of the error image data is necessary based on the information acquired in S502 and S503. That is, if the number of stored error image data corresponding to the combination of the model ID 403 and the format 601 of the error image data in which the analysis error has occurred is less than the maximum storage number 602 by one or more, it is determined that storage is necessary. Otherwise, it is determined that storage is unnecessary. For example, when the storage conditions shown in FIG. 4 are set, if the number of stored error image data generated by the model with the model ID 403 of "model_001" and the format 601 of JPG reaches 100, It is determined that storage is unnecessary. On the other hand, if the number of stored error image data whose format 601 is JPG is less than 100, it is determined that storage is necessary.

Note that the maximum storage number may be set for the total number of error image data to be stored regardless of the model and format of the imaging device. Furthermore, the storage condition set for the total number of error image data to be stored and the storage condition acquired in S503 may be combined. For example, even if the maximum storage number corresponding to the combination of the model ID 403 and the format 601 is not exceeded, if the total number of error image data to be stored exceeds the maximum value, it may be determined that storage is unnecessary. . Further, even if the maximum storage number corresponding to the combination of the model ID 403 and the format 601 is exceeded, if the total number of error image data does not exceed the maximum value, it may be determined that storage is necessary.

Further, even if the number of stored error image data exceeds the maximum storage number, it may be determined that storage is necessary by deleting the stored error image data. For example, among stored error image data, error image data with the oldest storage date and time, or error image data with a specific model ID 403 or format 601 may be deleted.

In S505, the error image storage necessity determination unit 105 sets storage unnecessary as the return value of the process, and the process ends.

In S506, the error image storage necessity determination unit 105 sets storage required as a return value of the process, and the process ends.

As described above, according to the image storage apparatus 101 of the present embodiment, among the image data in which an analysis error has occurred, the error image data is extracted based on the metadata such as the model of the imaging device and the attributes of the image data. It is possible to make a more detailed determination of the necessity of storage. As a result, wasteful storage of error image data can be avoided, the number of stored error image data can be suppressed, the cost required for storing error image data can be reduced, and storage can be performed more efficiently. Furthermore, in investigating the cause of an analysis error, the number of error image data to be investigated is narrowed down in advance, so that the investigation can proceed efficiently.

[Embodiment 2]
In the first embodiment, the example of determining whether or not to store data is determined based on the maximum stored number 602 of error image data in which an analysis error has occurred. In this embodiment, the need for storage is determined based on the model information of the imaging device that generated the image data in which the analysis error occurred and the error content of the analysis error. In addition, since this embodiment includes the same configuration as that of the first embodiment, differences from the first embodiment will be described below.

First, referring to FIGS. 7 and 9A, when the error image storage necessity determination unit 105 in this embodiment determines in S305 whether or not the error image data needs to be stored, the imaging device that generated the error image data will be described. Specific processing when using information will be described. FIG. 7 shows a flowchart of processing for determining whether or not to store error image data using information about the imaging device that generated the error image data. Also, FIG. 9A shows a storage condition table that designates information about an imaging device that generates error image data to be stored and used in this process.

In S<b>701 , the error image storage necessity determination unit 105 acquires from the image management table generated by the image registration unit 102 information about the imaging device that generated the error image data in which the analysis error occurred. Based on the image ID 401 of the error image data, the model ID 403, firmware version 404, and serial number 405 of the imaging apparatus that generated the error image data are obtained. If information about the imaging device cannot be acquired from the image management table generated by the image registration unit 102, the processing may be terminated at that point, or the items that cannot be acquired may be ignored or default values may be applied. Also good.

In S702, the error image storage condition management unit 106 acquires information about the imaging device that generates the error image data to be stored, which is the storage condition for the error image data. The error image storage condition management unit 106 uses the storage condition table illustrated in FIG. 9(a) to manage information about the imaging device that generates the error image data to be stored. In the storage condition table in this embodiment, values to be stored are stored with respect to the model ID 403 , firmware version 404 , and serial number 405 . If you do not set a specific value for each item, you can set a keyword such as "ALL" to indicate that all values apply, or simply leave the value blank. You can do it. Further, the conditions for the firmware version 404 and the serial number 405 may be conditions that specify a range of values instead of conditions that uniquely identify the target.

In S703, the error image storage necessity determination unit 105 determines whether storage of the error image data is necessary based on the information acquired in S701 and S702. That is, the error image storage necessity determining unit 105 determines whether or not the information about the imaging device acquired in S701 satisfies the storage conditions acquired in S702. For example, it is determined that error image data generated by an imaging apparatus having a model ID 403 of "model_001" and a firmware version 404 of "v1.2" needs to be stored.

Next, referring to FIGS. 8 and 9B, the failure information of the analysis error is used when the error image storage necessity determination unit 105 in this embodiment determines in S305 whether or not the error image data needs to be stored. The details of the specific processing in this case will be described. FIG. 8 shows a flowchart of processing for determining whether or not to store error image data using the analysis error failure information. FIG. 9(b) shows a storage condition table for designating error contents of error image data to be stored.

In S801, the image analysis unit 103 acquires error content from the image analysis unit 103 as failure information of the analysis error. As the content of the error, an error code 902 that distinguishes the type of error cause is acquired.

In S802, the error image storage condition management unit 106 acquires the error code of the error occurring in the error image data to be stored, which is the storage condition for the error image data. The error image storage condition management unit 106 uses the storage condition table illustrated in FIG. 9B to manage error codes 902 of errors occurring in error image data to be stored.

In S803, the error image storage necessity determination unit 105 determines whether storage of the error image data is necessary based on the information acquired in S801 and S802. That is, the error image storage necessity determining unit 105 determines whether or not the error code 902 acquired in S801 satisfies the storage condition acquired in S802. For example, if the error code 902 of the error generated in the error image data is "error_001", it is determined that storage is necessary, and if it is "error_002", it is determined that storage is unnecessary.

Note that the determination of whether or not storage is necessary in S305 is performed by combining information on the maximum storage number 602 of error image data shown in the first embodiment, information on the imaging apparatus shown in the present embodiment, and error content of the analysis error. You can For example, it may be determined that storage is necessary when all of the conditions are satisfied, or it may be determined that storage is necessary when some conditions are satisfied. Furthermore, it is also possible to set the priority of each condition and store only the error image data that satisfies the condition with the highest priority.

As described above, according to the image archiving apparatus of the present embodiment, it is possible to determine whether archiving is necessary based on the information about the imaging device that generated the image data in which the error occurred and the content of the analysis error. As a result, error image data can be stored based on the imaging device used and error content, and an error occurring in a specific model and specific error content can be investigated intensively.

[Embodiment 3]
In this embodiment, the user of the image archiving device can appropriately set the image conditions. In addition, since this embodiment includes the same configuration as the first and second embodiments, differences from the first embodiment will be described below.

FIG. 10 shows a functional block diagram of the image storage device in this embodiment.

The image archiving device 1001 in this embodiment further comprises error image archiving condition setting means 1002 and product information collecting means 1003 in addition to the image archiving devices 101 of the first and second embodiments shown in FIG. Error image storage condition setting means 1002 provides a user interface for the user to set storage conditions for image data. The product information collection unit 1003 collects new product information of the imaging device, update information of the firmware version 404, and the like. Then, based on the collected information, the image storage conditions are updated.

FIG. 11 shows a configuration example of a user interface provided by the error image storage condition setting means 1002. As shown in FIG.

The user interface displays a list of image data storage conditions. In the user interface, it is possible to add new storage conditions in addition to the displayed storage conditions. In this embodiment, the user interface displays a setting value list 1101 that is a candidate for each item of the storage condition, and the user selects a desired value from the list. A pointer 1102 is used for selection. Alternatively, the user interface may accept input of any value from the user without displaying the setting value list 1101 . Furthermore, the configuration may be such that the value of the storage condition that has already been set can be changed or the storage condition can be deleted.

Note that the storage condition items illustrated in FIG. 11 are merely examples, and settings of other items may be changed or added.

FIG. 12 shows an example of a storage condition change notice 1201 output by the product information collecting means 1003 in this embodiment. The storage condition change notification 1201 notifies the details of the storage condition change. The storage condition change notification 1201 includes storage condition change details 1202 and may further include a reason for changing the storage condition. For example, in the example of FIG. 12, the notification may include the release of a new model as a reason for changing the storage conditions.

The product information collection unit 1003 collects new product information of the imaging device, update information of the firmware version 404, etc., automatically creates storage conditions to be set, and uses the error image condition setting unit 1002 to store the created storage conditions. Register conditions.

The product information collection unit 1003 collects new product announcements posted on websites and firmware update notifications via the network, and detects product information updates. The product information collection means 1003 creates a storage condition based on the content of the detection when an update of the product information is detected. For example, when the release of a new imaging device is detected, a storage condition including the model ID 403 of the newly released imaging device is created and added. Further, when it is detected that the sales of an imaging device registered as a storage condition have been discontinued, the storage condition including the model ID 403 of the discontinued imaging device is deleted.

Note that the product information collecting means 1003 may have a table for managing product information, and add or update product information to the table. Further, when the storage conditions are changed, a procedure for approval by the user may be provided so that the changes are made only when the user approves.

As described above, according to the image archiving apparatus of the present embodiment, it is possible to appropriately set conditions for archiving error images. As a result, it is possible to appropriately change the storage conditions based on changes in product sales information and the intentions of the user of the image storage device.

In Embodiments 1 to 3, when error image data is stored in the error image storage unit 107, the entire image data may be stored as is, or only part of the image data may be stored. . For example, when storing moving image data, only a portion of the moving image data corresponding to a specific frame may be stored. Further, in the error image storage necessity judgment means 105, after judging necessity of storage, a step of approval by the user may be added so that the determination of necessity of storage can be corrected.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

105 error image storage necessity determination means 106 error image storage condition management means

Claims (10)

an extraction means for acquiring image data for which image analysis has failed in the image analysis means and extracting metadata from the image data;
when the metadata satisfies a predetermined storage condition, the image data extracted from the metadata is stored in a storage device for storage;
An information processing device comprising: the metadata is at least one of information about an imaging device that generated the image data, a format of the image data, and information about failure of the image analysis;
The information processing apparatus according to claim 1, characterized by: the predetermined storage condition includes information specifying the value of metadata possessed by the image data to be stored;
3. The information processing apparatus according to claim 1, wherein: the predetermined storage condition includes information specifying a combination of metadata values possessed by the image data to be stored;
4. The information processing apparatus according to claim 3, characterized by: the predetermined storage condition includes a maximum storage number of image data to be stored in the storage device;
The management means acquires the number of image data stored in the storage device, and if the number of stored image data is less than the maximum storage number, stores the image data from which the metadata has been extracted in the storage device. store in
5. The information processing apparatus according to any one of claims 1 to 4, characterized by: When the predetermined storage conditions include a plurality of storage conditions, the maximum storage number for each of the plurality of storage conditions is included.
6. The information processing apparatus according to claim 5, characterized by: 7. The information processing apparatus according to any one of claims 1 to 6, further comprising setting means for a user to set the predetermined storage conditions. 8. The information processing according to any one of claims 1 to 7, further comprising information collecting means for collecting information about the imaging device and updating the predetermined storage condition based on the collected information about the imaging device. Device. an extraction step of acquiring image data for which image analysis has failed in the image analysis means and extracting metadata from the image data;
a managing step of storing the image data from which the metadata is extracted in a storage device for archiving, if the metadata satisfies a predetermined storage condition;
An information processing method characterized by having A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 8.

Images