JP5942637B2 - Additional information management system, image sharing system, additional information management method, and program - Google Patents

Description

  The present invention relates to an invention for managing additional information that is not managed by an image management system.

  In recent years, an image taken by a digital camera or the like is uploaded by a user to an image management server on the Internet, while another user downloads the image from the image management server, so that a plurality of users can share the same image. Has been provided (see Patent Document 1).

  In recent years, SNS (Social Networking Service) such as Facebook has rapidly spread. With this SNS, while users can publish their comments, they can not only view other users' comments but also share image data among users.

  By the way, Exif (Exchangeable image file format), which is an image file format of digital cameras, automatically creates metadata along Exif when image data is saved as a JPEG (Joint Photographic Experts Group) file. It is common. Typical metadata includes shooting date / time, shutter speed, aperture, GPS information (eg longitude, latitude, altitude, direction), tilt angle of the imaging device with respect to the ground (eg roll direction, pitch) Direction, yaw direction rotation angle) and the like. For example, even when a user shoots a subject by tilting a digital camera, the user can view an image with the tilt corrected by correcting the image data using metadata indicating the tilt. .

  However, in many SNS image management servers including Facebook, metadata is considered to be privacy-related data, and the image management server separates and deletes metadata from uploaded image data, and images Only the data is managed. For this reason, even if the user downloads the image data from the image management server, the metadata is deleted, and thus there is a problem that the image processing such as correcting the image data using the metadata cannot be performed. Yes.

  The invention according to claim 1 manages image data and image identification information for identifying the image data in association with each other. The additional information added to the image data and used for image processing of the image data. An additional information management system that can communicate with an image management system that is not managed and that can communicate with a transmission device that transmits image data to which the additional information is added, and that manages the additional information, the transmission device The acquisition means for acquiring the additional information from the image data transmitted from the transmission apparatus, the transmission means for transmitting the image data transmitted from the transmission apparatus to the image management system, and the transmission means The image assigned to the image data in the image management system based on transmitting the image data to the image management system Receiving means for receiving different information from the image management system; storage means for storing the additional information acquired by the acquiring means and the image identification information received by the receiving means in association with each other. Is an additional information management system characterized by

  As described above, according to the present invention, the additional information management system receives the image identification information from the image management system while acquiring the additional information from the image data transmitted from the transmission device. Then, the additional information management system stores the additional information and the image identification information in association with each other. Thus, when the user downloads the image data from the image management system, the additional information can be downloaded from the additional information management system based on the corresponding image identification information. Therefore, image processing such as correcting the image data is performed. There is an effect that it can be performed.

1 is a schematic diagram of an image sharing system according to an embodiment of the present invention. (A) is a left side view of the photographing apparatus, (b) is a front view of the photographing apparatus, and (c) is a plan view of the photographing apparatus. It is the image figure which showed the use condition of an imaging device. (A) is the hemispherical image (front) image | photographed with the imaging device, (b) is the hemispherical image (back) image | photographed with the imaging device, (c) is the figure which showed the image represented by the Mercator projection. (A) is the figure which showed the part of the image represented by Mercator projection, and the image for share selection, (b) is the figure which showed the image for share selection. It is a figure of the picked-up image selection list | wrist which showed the image for a share selection. It is the figure which showed the omnidirectional panoramic image. It is the figure which showed the position of the virtual camera and predetermined | prescribed area | region when an omnidirectional panoramic image is made into a three-dimensional solid sphere. (A) is the three-dimensional perspective view of FIG. 7, (b) is a figure which shows the display apparatus with which the image of the predetermined area | region was displayed on the display. (A) is the figure which showed the predetermined area image in which each attention point was shown, (b) is the figure which showed the arbitrary area of interest and the surrounding predetermined area image. (A), (b) is the figure which showed the predetermined area | region image. It is a hardware block diagram of an imaging device. It is a hardware block diagram of a display apparatus. It is a hardware block diagram of an additional information management system and an image management system. It is the figure which showed the structure of the compression data file in the image file format specification for digital still cameras. It is a functional block diagram of a display device, an additional information management system, and an image management system according to the embodiment. It is a conceptual diagram which shows an additional information management table. It is a conceptual diagram which shows an image management table. FIG. 6 is a sequence diagram illustrating processing for uploading a captured image. It is the figure which showed the image object. FIG. 10 is a sequence diagram illustrating processing for downloading a captured image. It is the flowchart which showed the search process of additional information. It is the figure which showed the picked-up image when not correcting an image by 2D. It is the figure which showed the picked-up image when not correcting an image by 3D. FIG. 6 is a sequence diagram illustrating processing when a captured image cannot be downloaded.

  The embodiment of the present invention will be described below with reference to FIGS.

<< Summary of Embodiment >>
First, the outline of the present embodiment will be described with reference to FIGS. FIG. 1 is a schematic diagram of an image sharing system according to the present embodiment.

  As shown in FIG. 1, the image sharing system of the present embodiment is constructed by a photographing device 1, a plurality of display devices (3 a, 3 b), an additional information management system 5, and an image management system 7. Each display device (3a, 3b) is used by a user (A, B), respectively. Further, in the present embodiment, the case where the photographing apparatus 1 is operated by the user A is shown. In the following, an arbitrary display device among the plurality of display devices (3a, 3b) is represented as “display device 3”.

  The photographing apparatus 1 is a digital camera for obtaining an omnidirectional panoramic image. The display device 3 is a computer such as a smartphone, a tablet terminal, a notebook computer, a desktop computer, or a PDA (Personal Data Assistance). Furthermore, the additional information management system 5 and the image management system 7 are each a single or a plurality of server computers.

  In addition, the photographing device 1 can communicate with the display device 3 by a short-range wireless technology based on NFC (Near Field Communication) standard. Further, the display device 3 can communicate with the additional information management system 5 and the image management system 7 via the communication network 9. The communication network 9 is constructed by a wireless communication network using mobile communication systems such as 3G (3rd Generation) or WiMAX (Worldwide Interoperability for Microwave Access), base stations (9a, 9b, 9c), and the Internet. Note that wired communication may be performed between the imaging device 1 and the display device 3 and between the display device 3 and the communication network 9.

  Next, the external appearance of the imaging device 1 will be described with reference to FIG. 2A is a left side view of the photographing apparatus, FIG. 2B is a front view of the photographing apparatus, and FIG. 2C is a plan view of the photographing apparatus.

  As shown in FIG. 2A, the photographing apparatus 1 has a size that a human can hold with one hand. Further, as shown in FIGS. 2A, 2B, and 2C, at the upper part of the photographing apparatus 1, an image sensor 103a is provided on the front side (front side) and an image sensor is provided on the back side (rear side). 103b is provided. Further, as shown in FIG. 2B, an operation unit 115 such as a shutter button is provided on the front side of the photographing apparatus 1.

  Next, the usage situation of the imaging device 1 will be described with reference to FIG. FIG. 3 is an image diagram showing a usage state of the photographing apparatus. As shown in FIG. 3, the photographing device 1 is used for photographing a subject around the user with the hand held by the user. In this case, two hemispherical images can be obtained by imaging the subject around the user by the imaging device 103a and the imaging device 103b shown in FIG.

  Next, an image photographed by the photographing apparatus 1 and a synthesized image will be described with reference to FIG. 4A is a hemispheric image (front side) photographed by the photographing apparatus, FIG. 4B is a hemispheric image photographed by the photographing apparatus (rear side), and FIG. 4C is represented by Mercator projection. FIG. 6 is a diagram showing an image (hereinafter referred to as “Mercatl image”).

  As shown in FIG. 4A, the image obtained by the image sensor 103a is a hemispherical image (front side) curved by a fish-eye lens 102a described later. Also, as shown in FIG. 4B, the image obtained by the image sensor 103b is a hemispherical image (rear side) curved by a fish-eye lens 102b described later. Then, the hemispherical image (front side) and the hemispherical image inverted by 180 degrees (rear side) are combined by the photographing apparatus 1 to create a Mercator image as shown in FIG.

  Next, the relationship between the Mercator image and the shared selection image used for selecting the image will be described with reference to FIGS. 5 and 6. FIG. 5A is a diagram showing a portion of a Mercator image and a share selection image, and FIG. 5B is a diagram showing a share selection image. FIG. 6 is a view of a photographed image selection list showing the image for sharing selection.

  By using the portion indicated by the broken line on the Mercator image in FIG. 5A, the shared selection image in FIG. 5B is created. This share selection image is transmitted from the photographing apparatus 1 shown in FIG. 1 to the display apparatus 3a. Then, on the display 315 of the display device 3a, as shown in FIG. 6, a shared image selection list SL showing each share selection image CE is displayed. For example, a building in the commercial area is displayed as the share selection image CE, and the shooting date and time (September 20, 2011, 11:21), the image data size of the shot image (13.1 MB), and the attention point are set. The number (0) is displayed.

  Next, when the user A selects a desired share selection image CE, the display device 3a acquires a photographed image (Merkattle image) related to the selected share selection image CE from the photographing device 1.

  The acquired photographed image is shown in FIG. 7 from the Mercator image shown in FIG. 4C by using OpenGL ES (Open Graphics Library for Embedded Systems) in the display device 3a. A panoramic image is created. OpenGL ES is a graphics library used for visualizing 2D (2-Dimensions) and 3D (3-Dimensions) data. FIG. 7 is a diagram showing an omnidirectional panoramic image. The omnidirectional panoramic image may be a still image or a moving image.

  Next, a process of creating and displaying an image of a predetermined area (hereinafter referred to as “predetermined area image”) in the omnidirectional panoramic image will be described with reference to FIGS. FIG. 8 is a diagram illustrating the positions of the virtual camera and the predetermined area when the panoramic image is a three-dimensional solid sphere. 9A is a three-dimensional perspective view of FIG. 8, and FIG. 9B is a diagram showing a display device in which a predetermined area image is displayed on the display.

  If the omnidirectional panoramic image shown in FIG. 7 is a three-dimensional solid sphere CS, the virtual camera IC is located at the center of the omnidirectional panoramic image as shown in FIG. Three-axis rotation is possible: rotation from the center up and down, left and right, and rotation (tilt) from the viewpoint from this center. The predetermined area T in the omnidirectional panoramic image is specified by the position coordinates (x, y) and the angle of view of the virtual camera IC in the omnidirectional panoramic image. Thereby, the zoom of the predetermined area | region T can be expressed by expanding or contracting the range (arc) of a view angle.

  Then, as shown in FIG. 9A, the image of the predetermined region T in the omnidirectional panoramic image is displayed on the display 315 of the display device 3a as shown in FIG. 9B. It is displayed as a region image. The predetermined area image in this case is a partial image P showing a part of the omnidirectional panoramic image. On the display 315, a predetermined area image display area 3110 for displaying a predetermined area image, a thumbnail display area 3120 for displaying a thumbnail, and a comment display area 3130 for displaying a user's comment on the predetermined area image are displayed.

  Furthermore, the captured image shown in FIG. 4C is uploaded to the image management system 7 via the communication network 9 from the display device 3a as a transmission device. Then, for example, the user B uses the display device 3 b as a receiving device to download the captured image from the image management system 7 via the communication network 9. Thereby, each user can share a captured image.

  Also, as shown in FIG. 10, in the image sharing system of the present embodiment, it is possible to set a point of interest that each user is paying attention to the captured image. FIG. 10A is a diagram showing a predetermined area image in which each attention point is shown, and FIG. 10B is a diagram showing an arbitrary attention point and surrounding predetermined area images.

  As shown in FIG. 10A, on the display 315 of the display device 3b, a plurality of attention points set by attention by a single or a plurality of users using the image sharing system of the present embodiment. (AP1, AP2, AP3, AP4) is displayed. Further, images of a predetermined area T within a certain range including each point of interest (AP1, AP2, AP3, AP4) are displayed in the state of thumbnails (T1, T2, T3, T4) at the bottom of the display 315, respectively. . When the user B selects the thumbnail T1, for example, as shown in FIG. 10B, a predetermined region image (here, the partial image P1) that is the original image of the thumbnail T1 is displayed on the display 315. Will be displayed.

  It is also possible to set a point of interest for the predetermined area image shown in FIG. In this case, a thumbnail related to the predetermined area image including the set attention point is displayed.

  In the above description, the display device 3a and the display device 3b are described as the transmission device, but the present invention is not limited thereto, and the display device 3a may be used as the reception device and the display device 3b may be used as the transmission device. In other words, the display device 3a can also download a photographed image from the image management system 7, and the attention point setting as shown in FIG. 10 and the image as shown in FIG. Can be made.

<< Hardware Configuration of Embodiment >>
Next, the hardware configurations of the imaging apparatus, display apparatus, additional information management system, and image management system of this embodiment will be described in detail with reference to FIGS.

  First, the hardware configuration of the photographing apparatus 1 will be described with reference to FIG. FIG. 11 is a hardware configuration diagram of the photographing apparatus. In the following, the photographing apparatus 1 is an omnidirectional photographing apparatus using two imaging elements, but the number of imaging elements may be three or more. In addition, it is not always necessary to use an apparatus dedicated to omnidirectional imaging. By attaching a retrofit omnidirectional imaging unit to an ordinary digital camera, a smartphone, or the like, the apparatus may have substantially the same function as the imaging apparatus 1. .

  As shown in FIG. 11, the imaging apparatus 1 includes an imaging unit 101, an image processing unit 104, an imaging control unit 105, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, and an SRAM (Static Random Access). Memory (RAM) 113, DRAM (Dynamic Random Access Memory) 114, operation unit 115, network I / F 116, communication unit 117, and antenna 117a.

  Among these, the imaging unit 101 includes wide-angle lenses (so-called fish-eye lenses) 102a and 102b each having an angle of view of 180 ° or more for forming a hemispherical image, and two imaging units provided corresponding to the wide-angle lenses. Elements 103a and 103b are provided. The image sensors 103a and 103b are image sensors such as a CMOS (Complementary Metal Oxide Semiconductor) sensor and a CCD (Charge Coupled Device) sensor that convert an optical image obtained by a fisheye lens into image data of an electric signal and output the image data. A timing generation circuit for generating a vertical synchronization signal, a pixel clock, and the like, and a register group in which various commands and parameters necessary for the operation of the image sensor are set.

  The imaging elements 103a and 103b of the imaging unit 101 are each connected to the image processing unit 104 via a parallel I / F bus. On the other hand, the imaging elements 103 a and 103 b of the imaging unit 101 are connected to a serial I / F bus (I2C bus or the like) separately from the imaging control unit 105. The image processing unit 104 and the imaging control unit 105 are connected to the CPU 111 via the bus 110. Further, ROM 112, SRAM 113, DRAM 114, operation unit 115, network I / F 116, communication unit 117, and electronic compass 118 are connected to the bus 100.

  The image processing unit 104 takes in the image data output from the image sensors 103a and 103b through the parallel I / F bus, performs predetermined processing on the respective image data, and then combines these image data. Then, data of a Mercator image as shown in FIG.

  In general, the imaging control unit 105 sets a command or the like in a register group of the imaging elements 103a and 103b using the I2C bus with the imaging control unit 105 as a master device and the imaging elements 103a and 103b as slave devices. Necessary commands and the like are received from the CPU 111. The imaging control unit 105 also uses the I2C bus to capture status data of the register groups of the imaging elements 103a and 103b and send it to the CPU 111.

  The imaging control unit 105 instructs the imaging elements 103a and 103b to output image data at the timing when the shutter button of the operation unit 115 is pressed. Some photographing apparatuses have a preview display function by a display and a function corresponding to a moving image display. In this case, output of image data from the image sensors 103a and 103b is continuously performed at a predetermined frame rate (frame / min).

  Further, as will be described later, the imaging control unit 105 also functions as a synchronization control unit that synchronizes the output timing of image data of the imaging elements 103a and 103b in cooperation with the CPU 111. In the present embodiment, the photographing apparatus is not provided with a display unit, but a display unit may be provided.

  The CPU 111 controls the overall operation of the photographing apparatus 1 and executes necessary processes. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are work memories, and store programs executed by the CPU 111, data being processed, and the like. In particular, the DRAM 114 stores image data being processed by the image processing unit 104 and processed Mercator image data.

  The operation unit 115 is a general term for various operation buttons, a power switch, a shutter button, a touch panel that has both display and operation functions, and the like. The user inputs various shooting modes and shooting conditions by operating the operation buttons.

  The network I / F 116 is a general term for an interface circuit (USB I / F or the like) with an external medium such as an SD card or a personal computer. Further, the network I / F 116 may be a network interface regardless of wireless or wired. The data of the Mercator image stored in the DRAM 114 is recorded on an external medium via the network I / F 116 or, if necessary, via the network I / F 116 serving as the network I / F, the display device 3. To the external device.

  The communication unit 117 communicates with an external device such as the display device 3 through an antenna 117a provided in the photographing device 1 by a short-range wireless technology such as WiFi (wireless fidelity) or NFC. The communication unit 117 can also transmit Mercator image data to an external device of the display device 3.

  The electronic compass 118 calculates the azimuth and tilt (Roll rotation angle) of the photographing apparatus 1 from the earth's magnetism and outputs azimuth / tilt information. This azimuth / tilt information is an example of metadata along Exif, and is used for image processing such as image correction of a captured image. In the following description, “additional information” is used as a broad concept including metadata.

  Next, the hardware configuration of the display device 3 will be described with reference to FIG. FIG. 12 is a hardware configuration diagram when the display device is a smartphone.

  As shown in FIG. 12, the display device 3 includes a CPU 301 that controls the operation of the entire display device 3, a ROM 302 that stores basic input / output programs, and a RAM (Random Access Memory) 303 that is used as a work area for the CPU 301. , An EEPROM (Electrically Erasable and Programmable ROM) 304 that reads or writes data according to the control of the CPU 301, a CMOS sensor 305 as an image sensor that captures an image of the subject and obtains image data according to the control of the CPU 301, and an electronic magnetic compass that detects geomagnetism , A gyro compass, various acceleration / direction sensors 306 such as an acceleration sensor, and a media drive 308 that controls reading or writing (storage) of data with respect to a recording medium 307 such as a flash memory. In accordance with the control of the media drive 308, the recording medium 307 from which data that has already been recorded is read or to which data is newly written and stored is detachable.

  The EEPROM 304 stores an operating system (OS) executed by the CPU 301, other programs, and various data. A CCD sensor may be used instead of the CMOS sensor 305.

  Furthermore, the display device 3 includes a voice input unit 311 that converts voice into a voice signal, a voice output unit 312 that converts voice signals into voice, an antenna 313a, and a wireless communication signal that uses the antenna 313a to obtain the nearest base station. A GPS signal including position information (latitude, longitude, and altitude) of the display device 3 by a communication unit 313 that communicates with the station 9a, a GPS (Global Positioning Systems) satellite, or IMES (Indoor MEssaging System) as an indoor GPS. A GPS receiver 314 for receiving, a display 315 such as a liquid crystal display or an organic EL display for displaying an image of a subject, various icons, and the like, and a pressure-sensitive or electrostatic panel placed on the display 315 and configured by a finger, a touch pen, etc. A touch panel 316 for detecting a touch position on the display 315 by touching with the And it includes an address bus, a data bus, and the like of the bus line 310 for electrically connecting the respective units.

  Note that the voice input unit 311 includes a microphone that inputs voice, and the voice output unit 312 includes a speaker that outputs voice.

  Next, the hardware configuration of the additional information management system 5 and the image management system 7 will be described with reference to FIG. FIG. 13 is a hardware configuration diagram of the additional information management system 5 and the image management system 7. Since both the additional information management system 5 and the image management system 7 are general server computers, the configuration of the additional information management system 5 will be described below, and the description of the configuration of the image management system 7 will be omitted.

  The additional information management system 5 includes a CPU 501 that controls the operation of the additional information management system 5 as a whole, a ROM 502 that stores programs used to drive the CPU 501 such as an IPL, a RAM 503 that is used as a work area for the CPU 501, and the additional information management system 5 HD 504 for storing various data such as programs for data, HDD (Hard Disk Drive) 505 for controlling the reading or writing of various data to the HD 504 according to the control of the CPU 501, data reading or writing to a recording medium 506 such as a flash memory ( Media drive 507 for controlling storage, a display 508 for displaying various information such as a cursor, menu, window, character, or image, and a network for data communication using the communication network 9 / F509, keyboard 511 having a plurality of keys for inputting characters, numerical values, various instructions, etc., mouse 512 for selecting and executing various instructions, selecting a processing target, moving the cursor, etc., removable recording medium As an example, a CD-ROM drive 514 that controls reading or writing of various data with respect to a CD-ROM (Compact Disc Read Only Memory) 513, and the above-described components are electrically connected as shown in FIG. A bus line 510 such as an address bus or a data bus for connection is provided.

  FIG. 14 is a diagram showing the structure of a compressed data file in the digital still camera image file format standard. The image data obtained by the photographing apparatus 1 has a compressed data file structure as shown in FIG. This is shown in “Exif 2.3, an image format standard for digital cameras” (http://www.cipa.jp/hyoujunka/kikaku/pdf/DC-008-2010_E.pdf) established on April 26, 2010. This is a compressed data file structure. Of these, APP1 indicates an application data segment 1 used for recording metadata for an image. In APP1, an EXIF IFD that can store unique metadata (Maker Note) for each device manufacturer such as a digital camera is constructed. In the present embodiment, image type information indicating the type of image and azimuth / tilt information calculated by the electronic compass 118 are shown as additional information. The image type information is, for example, an identification code indicating that the image is an omnidirectional panoramic image, or an identification code indicating that the image is a normal planar image.

<< Functional Configuration of Embodiment >>
Next, the functional configuration of the present embodiment will be described with reference to FIG. FIG. 15 is a functional block diagram of the display device 3, the additional information management system 5, and the image management system 7 that constitute a part of the image sharing system of the present embodiment. In FIG. 15, the display device 3, the additional information management system 5, and the image management system 7 can perform data communication via the communication network 9.

<Functional configuration of display device>
As shown in FIG. 15, the display device 3 includes a transmission / reception unit 31, an operation input reception unit 32, a creation unit 33, a display control unit 34, an image processing unit 38, and a storage / extraction unit 39. . Each of these units is a function realized by any one of the constituent elements shown in FIG. 11 being operated by a command from the CPU 111 according to a program for the display device 3 expanded from the SRAM 113 to the DRAM 114, or Means.

  In addition, the display device 3 includes a storage unit 3000 configured by the ROM 112, the SRAM 113, and the DRAM 114 illustrated in FIG.

(Functional configuration of display device)
Next, each functional configuration of the display device 3 will be described in more detail with reference to FIGS. 12 and 15.

  The transmission / reception unit 31 of the display device 3 is realized mainly by the processing of the communication unit 313 and the CPU 301 shown in FIG. 12, and the additional information management system 5 or the image management system 7 and various data via the communication network 9. (Or information) is sent and received.

  The operation input receiving unit 32 is realized mainly by the processing by the touch panel 316 and the CPU 301, and receives various selections or inputs from the user.

  The creation unit 33 is realized mainly by the processing of the CPU 301, and based on the user's designation or selection received by the operation input reception unit 32, as shown in FIG. Create an image. Further, the creation unit 33 creates the thumbnail shown in FIG. 10 based on the predetermined area image.

  The display control unit 34 is realized mainly by the processing of the CPU 301 and performs control for displaying various images, characters, and the like on the display 315. For example, the display control unit 34 causes the image data processed by the image processing unit 38 to be displayed on the display 315 of the display device 3. Further, the display control unit 34 causes the display 315 to display a predetermined area image related to the thumbnail selected by the operation input receiving unit 32.

  The image processing unit 38 performs image processing on the image data using the additional information. This image processing is image editing, image processing, or image correction (correction). The image processing unit 38 performs speech balloon addition, text addition, trimming, resizing, and the like as image editing. The image processing unit 38 performs image cropping, sharpening, embossing, blurring, or the like as image processing. Further, the image processing unit 38 performs red-eye correction, brightness adjustment, contrast adjustment, azimuth correction, inclination correction, and the like as image correction.

  The storage / extraction unit 39 stores various data (or information) in the storage unit 3000 and extracts various data (or information) from the storage unit 3000. The storage / extraction unit 39 records, for example, each data of a captured image, a shared selection image, or a predetermined area image in the storage unit 3000 or extracts from the storage unit 3000.

<Functional configuration of additional information management system>
Next, each functional configuration of the additional information management system 5 will be described in detail with reference to FIG. The additional information management system 5 includes a transmission / reception unit 51, a determination unit 52, an acquisition unit 55, and a storage / extraction unit 59. Each of these components is realized by any one of the constituent elements shown in FIG. 13 operating according to a command from the CPU 501 according to the program for the additional information management system 5 expanded from the HD 504 onto the RAM 503. It is a function or means.

  Further, the additional information management system 5 includes a storage unit 5000 constructed by the RAM 503 and the HD 504 shown in FIG. In the storage unit 5000, an additional information management DB 5001 configured by an additional information management table described later is constructed.

(Additional information management table)
FIG. 16 is a conceptual diagram showing an additional information management table. In this additional information management table, user identification information for identifying a user, image identification information for identifying image data, and the above-described additional information are stored and managed in association with each other.

  In FIG. 16, a user ID (Identification) is shown as an example of the user identification information. However, if the user can be specified, the user ID is not necessary. For example, the user identification information includes a user name, an employee number, a student ID number, a national number based on a national gross number system, and the like. In FIG. 16, an image ID is shown as an example of the image identification information. However, if the image data can be specified, it is not necessary to be an image ID. For example, the image identification information includes the file name of the image data.

(Functional configuration of additional information management system)
Next, each functional configuration of the additional information management system 5 will be described in detail with reference to FIGS. 13 and 15.

  The transmission / reception unit 51 of the additional information management system 5 is realized mainly by the processing of the network I / F 509 and the CPU 501 shown in FIG. 13, and is connected to the display device 3 or the image management system 7 via the communication network 9. Send and receive data (or information).

  The determination unit 52 is realized mainly by the CPU 501 shown in FIG. 13 and determines whether or not additional information corresponding to the image identification information transmitted from the display device 3 is stored in the additional information management table. to decide.

  The acquisition unit 55 acquires additional information added to the image data from the image data stored in the storage unit 5000 after being received by the transmission / reception unit 51. Note that, by this acquisition, the additional image information may remain added to the original image data, or the additional image information may not be added to the original image data (separated state). Good.

  The storage / extraction unit 59 stores various data (or information) in the storage unit 5000 and extracts various data (or information) from the storage unit 5000.

<Functional configuration of image management system>
Next, the functional configuration of the image management system 7 will be described in detail with reference to FIGS. 13 and 15. The image management system 7 includes a transmission / reception unit 71, a creation unit 72, and a storage / extraction unit 79. Each of these units is a function realized by any one of the constituent elements shown in FIG. 13 operating according to a command from the CPU 501 according to the program for the image management system 7 developed from the HD 504 onto the RAM 503. Or means.

  Further, the image management system 7 has a storage unit 7000 constructed by the RAM 503 and the HD 504 shown in FIG. In the storage unit 7000, an image management DB 7001 configured by an image management table described later is constructed.

(Image management table)
FIG. 17 is a conceptual diagram showing an image management table. In this image management table, image object data is managed. This image object indicates user identification information for identifying a user, image identification information for identifying an image, and a URL (Uniform Resource Locator) that is a storage location in the storage unit 7000 of image data related to the image identification information. URL information and attribute information are managed in association with each other.

  Note that the user identification information and the image identification information of the image object are the same as the user identification information and the image identification information shown in FIG. On the other hand, the attribute information is different from the additional information shown in FIG. That is, the attribute information is original information (image file name, information indicating the number of vertical and horizontal pixels of the image, etc.) of the image data created by the creation unit 72 described later based on the image data, It is not used for image processing of data.

(Each functional configuration of the image management system)
Next, each functional configuration of the image management system 7 will be described in detail with reference to FIG.

  The transmission / reception unit 71 of the image management system 7 is realized mainly by the processing of the network I / F 509 and the CPU 501 shown in FIG. 13, and various kinds of information are displayed with the display device 3 or the additional information management system 5 via the communication network 9. Send and receive data (or information).

  The creation unit 72 assigns image identification information to the image data in order to manage the image data sent from the display device 3 by the image management system 7, and assigns the image identification information to the header portion of the image data. To do. The creation unit 72 creates an image object.

  The storage / extraction unit 79 stores various data (or information) in the storage unit 7000 and extracts various data (or information) from the storage unit 5000.

<< Processing or Operation of Embodiment >>
Next, a process in which the user A uploads image data using the display device 3a will be described with reference to FIGS. FIG. 18 is a sequence diagram illustrating processing for uploading a captured image. FIG. 19 shows an image object.

  Note that the communication protocol via the communication network 9 between the display device 3a, the additional information management system 5, and the image management system 7 is HTTP communication based on the hypertext communication protocol (HTTP). The display device 3a corresponds to an HTTP client, and the additional information management system 5 and the image management system 7 correspond to an HTTP server.

  Unless otherwise specified, the arrows from the display device 3a to the additional information management system 5 and the image management system 7 in each sequence diagram correspond to HTTP requests, and from the additional information management system 5 and the image management system 7 to the display device 3a. The arrow corresponds to an HTTP response.

  First, the user A captures the image data to which the additional information is added from the photographing device 1 into the storage unit 3000 of the display device 3a.

  Next, when the user A selects the image data to be uploaded, the operation input reception unit 32 of the display device 3a receives the selection of the image data to be uploaded (step S1). Then, the transmission / reception unit 31 of the display device 3a sends the user identification information of the user A, the image data to which the additional information is added, and the file name data of the image data to the additional information management system 5 via the communication network 9. To upload the image data (step S2). As a result, the transmission / reception unit 51 of the additional information management system 5 receives the user identification information and the image data to which the additional information is added.

  Next, the acquisition unit 55 of the additional information management system 5 acquires additional information from the image data downloaded in step S1 (step S3). Then, the storage / extraction unit 59 associates the user identification information received in step S2 with the additional information acquired in step S3, and stores and manages it in the additional information management table (see FIG. 16) (see FIG. 16). Step S4).

  Next, the transmitting / receiving unit 51 transmits the image data and user identification information received in step S2 to the image management system 7 via the communication network 9 (step S5). Thereby, the transmission / reception unit 71 of the image management system 7 receives the image data and the user identification information.

  Next, the creation unit 72 of the image management system 7 creates an image object as shown in FIG. 19 based on the image data received in step S5 (step S6). The image object data includes user identification information, image identification information, URL information, and attribute information.

  Next, the storage / extraction unit 79 stores and manages the image object created in step S6 in the image management table (see FIG. 17) (step S7). Then, the transmission / reception unit 71 transmits the image object to the additional information management system 5 via the communication network 9 (step S8). Thereby, the transmitting / receiving unit 51 of the additional information management system 5 receives the image object.

  The storage / extraction unit 59 of the additional information management system 5 stores the image identification information of the image object received in step S8 in association with the user identification information and additional information stored in step S4. (Step S9). Thereby, the additional information management table as shown in FIG. 16 is constructed.

  Then, the transmission / reception unit 51 transmits the image identification information together with the result indicating that the management of the additional information is completed to the display device 3a via the communication network 9 (step S10). Thus, a series of image data upload processing is completed.

  Next, a process in which the user B downloads image data using the display device 3b will be described with reference to FIGS. FIG. 20 is a sequence diagram illustrating a process of downloading a captured image.

  First, the user B requests a file name list indicating file names of images to be downloaded from the transmission / reception unit 31 of the display device 3b to the image management system 7 via the communication network 9 (step S21). Thereby, the transmission / reception unit 71 of the image management system 7 accepts a request for a file name list.

  In response to this, the storage / extraction unit 79 of the image management system 7 reads the image identification information and the file name information of the additional information from the image management table (see FIG. 17) (step S22). Then, the transmission / reception unit 71 transmits image identification information and file name list data including file name information corresponding to the image identification information to the display device 3b via the communication network 9 (step S23). Thereby, the transmission / reception unit 31 of the display device 3b receives the data of the file name list.

  Next, when the user B selects a file name indicating image data to be downloaded from the file name list, the operation input receiving unit 32 of the display device 3b receives a file name selection (step S24). Then, the transmission / reception unit 31 of the display device 3b transmits the image identification information corresponding to the file name accepted in step S24 to the additional information management system 5 via the communication network 9 and requests additional information (step). S25). As a result, the transmission / reception unit 51 of the additional information management system 5 receives the image identification information and receives a request for additional information.

  Next, the additional information management system 5 performs a process for searching for additional information (step S26). This search process will be described with reference to FIG. FIG. 21 is a flowchart showing the additional information search process.

  First, as shown in FIG. 21, the storage / extraction unit 59 searches the additional information management table (see FIG. 16) using the image identification information received in step S25 as a search key (step S101). .

  Next, the determination unit 52 determines whether or not corresponding additional information is stored in step S101 (step S102). If the determination unit 52 determines that the corresponding additional information is stored (YES), the storage / extraction unit 59 reads the corresponding additional information, and then continues the processing of step S27 in FIG. Is done. On the other hand, when the determination unit 52 determines that the corresponding additional information is not stored (NO), the process of step S41 of FIG. 24 described later is performed.

  Next, when it is determined in step S102 that the corresponding additional information is stored (YES), the description is continued from the process of step S27 in FIG. As shown in FIG. 20, the transmitting / receiving unit 51 transmits additional information to the display device 3b via the communication network 9 (step S27). Thereby, the transmission / reception unit 31 of the display device 3b receives the additional information.

  Next, the display device 3b transmits the image identification information of the image that the user B wants to download to the image management system 7 via the communication network 9, and requests an image object (step S28). Thereby, the transmission / reception unit 71 of the image management system 7 receives the image identification information and accepts the request for the image object.

  Next, the storage / extraction unit 79 of the image management system includes the image identification information by searching the image management table (see FIG. 17) using the image identification information received in step S28 as a search key. An image object is extracted (step S29). Then, the transmission / reception unit 71 transmits the data of the image object extracted in step S29 to the display device 3b via the communication network 9 (step S30). Thereby, the transmission / reception unit 31 of the display device 3b receives the data of the image object.

  Next, the transmission / reception unit 31 requests the image data managed by the URL from the image management system 7 by accessing the URL included in the image object (step S31). In response to this, the storage / extraction unit 79 of the image management system 7 extracts the requested image data from the storage unit 7000 (step S32). Then, the transmission / reception unit 79 transmits the image data extracted in step S32 to the display device 3b via the communication network 9 (step S33). Thereby, the transmission / reception unit 31 of the display device 3b receives the image data.

  Next, the image processing unit 38 of the display device 3b performs image processing on the image data received in step S33 based on the additional information received in step S27 (step S34). The display device 3b displays the image data after image processing on the display 315 (step S35).

  Here, image correction for correcting the horizon as an example of image processing will be described with reference to FIGS. Note that FIG. 22 is a diagram illustrating a captured image in 2D when image correction is not performed. FIG. 23 is a diagram illustrating a captured image in 3D when image correction is not performed.

  For example, when the user A captures an image of a subject with the image capturing device 1 and the image capturing device 1 is tilted, the image is not corrected by the display device 3b. FIG. 22 (in the case of 2D) Alternatively, the image remains tilted as shown in FIG. 23 (when displayed in 3D). However, when the display control unit 34 visualizes the 2D image data shown in FIG. 22 into 3D, the image processing unit 38 corrects the display position based on the tilt information of the azimuth / tilt information. Thereby, the display control part 34 can be made to display on the display 315 as an omnidirectional panoramic image which leveled the horizon, as FIG. 7 shows. After that, when the user B touches the display 315, the operation input receiving unit 32 receives the touch, and the display control unit 34 can display a predetermined area image as shown in FIG. 9B.

  Next, with reference to FIG. 24, processing after it is determined in step S102 that additional information is not stored will be described. Note that FIG. 24 is a sequence diagram illustrating a process when a captured image cannot be downloaded.

  As shown in FIG. 24, the transmission / reception unit 51 of the additional information management system 5 transmits an error message indicating that the corresponding additional information is not stored to the display device 3b via the communication network 9. Thereby, the transmission / reception unit 31 of the display device 3b receives the error message.

  Next, the display control unit 34 of the display device 3b displays an error message on the display 315 (step S42). As described above, when the additional information management system 5 does not manage the additional information, the download process can be omitted by not proceeding to the process of downloading the image data (steps S31 to S33). . As a use case in which this error message is displayed, a case where the image is a normal 2D image that is not a Mercator image can be considered. When the photographed image is a Mercator image, even if it is displayed as an omnidirectional panoramic image visualized in 3D as shown in FIG. 7, it is displayed without a sense of incongruity, but a normal 2D image that is not a Mercator image is displayed in 3D. If it is visualized and displayed, it will become increasingly curved and become very difficult to see. Therefore, in the present embodiment, the process of downloading image data is not allowed to proceed unless additional information that contributes to image correction is stored.

  Note that even when the additional information is not stored, the image data download process may be performed. In this case, the display device 3 may suppress the visualization to display in 3D and perform normal 2D display.

<< Main effects of this embodiment >>
As described above, according to the present embodiment, the additional information management system 5 acquires the additional information from the image data transmitted from the display device 3a (transmission device), while the image management system 7 identifies the image. Receive information. The additional information management system 5 can manage the additional information in place of the image management system 7 by storing the additional information and the image identification information in association with each other. Thereby, when the user B downloads image data from the image management system 7 by the display device 3b (receiving device), the additional information can be downloaded from the additional information management system 5 based on the corresponding image identification information. There is an effect that image processing such as correction of image data can be performed.

[Supplement of Embodiment]
The additional information management system 5 and the image management system 7 in each of the above embodiments may be constructed by a single computer, or a plurality of units arbitrarily assigned by dividing each unit (function, means, or storage unit). It may be constructed by a computer.

  In addition, a recording medium such as a CD-ROM in which the programs of the above-described embodiments are stored, and the HD 504 in which these programs are stored may be provided domestically or abroad as a program product. it can.

1 Imaging device 3a Display device (an example of a transmission device)
3b Display device (an example of a receiving device)
DESCRIPTION OF SYMBOLS 5 Additional information management system 7 Image management system 9 Communication network 9a, 9b Base station 31 Transmission / reception part (an example of a transmission means, an example of a reception means)
32 Operation input reception part 33 Creation part (an example of creation means)
34 Display control unit (an example of display control means)
38 Image processing unit (an example of image processing means)
39 storage / extraction unit 51 transmission / reception unit 52 determination unit 55 acquisition unit (an example of an acquisition unit)
71 Transmission / Reception Unit 72 Creation Unit 79 Storage / Extraction Unit 315 Display (an example of display means)
3000 storage unit (an example of storage means)
5000 storage unit 5001 additional information management DB
7000 storage unit 7001 image management DB

JP 2011-120201 A

Claims (11)

Manages image data and image identification information for identifying the image data in association with each other, but can communicate with an image management system that is added to the image data and does not manage additional information used for image processing of the image data And an additional information management system that can communicate with a transmission device that transmits image data to which the additional information is added, and that manages the additional information,
Obtaining means for obtaining the additional information from the image data transmitted from the transmission device;
Transmission means for transmitting the image data transmitted from the transmission device to the image management system;
Receiving means for receiving from the image management system the image identification information assigned to the image data in the image management system based on transmitting the image data to the image management system by the transmission means;
Storage means for associating and storing the additional information acquired by the acquisition means and the image identification information received by the receiving means;
An additional information management system comprising: The reception means receives user identification information for identifying a user of the transmission device in addition to the image data from the transmission device,
The additional information management system according to claim 1, wherein the storage unit stores the user identification information in association with the additional information and the image identification information.   The additional information according to claim 1, wherein the transmission unit transmits the image identification information to the transmission device when the reception unit receives the image identification information from the image management system. Management system. The receiving means receives the image identification information from a predetermined receiving device and receives the request for the additional information,
An extraction unit that extracts the corresponding additional information by searching the storage unit based on the image identification information received by the reception unit;
The additional information management system according to any one of claims 1 to 3, wherein the transmitting unit transmits the additional information extracted by the extracting unit to the receiving device. Determining means for determining whether there is corresponding additional information by searching the storage means based on the image identification information received by the receiving means;
When the determining means determines that there is corresponding additional information, the transmitting means transmits the corresponding additional information to the receiving device, and when the determining means determines that there is no corresponding additional information 5. The additional information management system according to claim 4, wherein the transmitting means transmits an error message to the receiving device.   The additional information management system according to any one of claims 1 to 5, wherein the image processing is image editing, image processing, or image correction.   The additional information management system according to claim 1, wherein the image data is Mercator image data. The additional information management system according to any one of claims 1 to 7,
The transmitting device or the receiving device;
An image sharing system comprising:   The image sharing system according to claim 8, wherein the image sharing system includes the image management system. Manages image data and image identification information for identifying the image data in association with each other, but can communicate with an image management system that is added to the image data and does not manage additional information used for image processing of the image data And an additional information management method executed by an additional information management system that manages the additional information, and is capable of communicating with a transmission device that transmits image data to which the previous additional information is added.
The additional information management system is
An acquisition step of acquiring the additional information from the image data transmitted from the transmission device;
Transmitting the image data received from the transmission device to the image management system;
Receiving the image identification information assigned to the image data in the image management system based on transmitting the image data to the image management system in the transmitting step from the image management system;
A storage step of associating the additional information acquired by the acquisition step with the image identification information received by the reception step and storing the associated information in a predetermined storage unit;
The additional information management method characterized by performing this.   A program causing a computer to execute each step according to claim 10.

Images