JP6508288B2 - SYSTEM, IMAGE SHARING SYSTEM, COMMUNICATION METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to an invention for managing additional information not managed by an image management system.

  In recent years, while an image taken by a digital camera or the like is uploaded to an image management server on the Internet by one user, another user downloads the image from the image management server, whereby a plurality of users have the same image. There is provided a system that shares the two (see Patent Document 1).

  Also, in recent years, SNS (Social Networking Service) such as Facebook is rapidly spreading. This SNS allows not only the user to view his / her comment but also to view the comments of other users, as well as sharing image data among the users.

  By the way, Exif (Exchangeable image file format), which is an image file format of a digital camera, automatically creates metadata along Exif when image data is saved as a JPEG (Joint Photographic Experts Group) file. It is common. Representative metadata includes shooting date, shutter speed, aperture, GPS information (for example, longitude, latitude, altitude, azimuth), an inclination angle of the imaging device with respect to the ground (for example, roll direction, pitch) The direction, the rotation angle in the yaw (Yaw) direction, and the like are included. For example, even when the user tilts the digital camera and captures an object, the user can view the image whose tilt has been corrected by correcting the image data using the metadata indicating the tilt. .

However, in many SNS image management servers (eg, systems) , including Facebook, metadata is considered to be privacy related data, and the image management server separates metadata from uploaded image data. Delete and manage only image data. Therefore, even if the user downloads the image data from the image management server, the metadata is deleted, and a problem arises that the image processing such as correction of the image data can not be performed using the metadata. There is.

The invention according to claim 1 is a system capable of communicating with a transmitting device that transmits image data to which additional information used for image processing is added, and a receiving unit that receives the image data from the transmitting device A first storage unit for storing the received image data and image identification information for identifying the image data in association with each other; the additional information added to the received image data; and the image It is a system characterized by having the 2nd storage means which associates identification information and stores it, and the transmission means which transmits the picture identification information to the transmitting device .

As described above, according to the present invention, the system stores the image identification information for identifying the image data transmitted from the transmission device in the first storage unit, while the system transmits the image identification information from the transmission device. The additional information added to the incoming image data and the image identification information stored in the first storage means are associated and stored in the second storage means. Effect Thus, when the user downloads the image data from the system, it is possible to download-added information on the basis of the corresponding image identification information, that the image data it is possible to perform image processing such as correcting the Play.

FIG. 1 is a schematic view of an image sharing system according to an embodiment of the present invention. (A) is a left side view of the imaging device, (b) is a front view of the imaging device, and (c) is a plan view of the imaging device. It is an image figure showing the use situation of an imaging device. (A) is a hemispherical image (front) taken by the imaging device, (b) is a hemispherical image (after) taken by the imaging device, and (c) is an image represented by the Mercator projection. (A) is a figure which showed the part of the image represented by the Mercator projection method and the image for share selection, (b) is a figure which showed the image for share selection. It is a figure of the picked-up image selection list | wrist which showed the image for share selection. It is a figure showing an omnidirectional panoramic image. It is the figure which showed the position of the virtual camera and the predetermined area | region at the time of making an omnidirectional panoramic image into a three-dimensional solid sphere. (A) is a three-dimensional perspective view of FIG. 7, (b) is a figure which shows the display apparatus with which the image of the predetermined area | region was displayed on the display. (A) is a figure which showed the predetermined area | region image in which each attention point was shown, (b) is a figure which showed the arbitrary attention point and the predetermined area | region image of a periphery. (A), (b) is the figure which showed the predetermined area | region image. It is a hardware block diagram of an imaging device. It is a hardware block diagram of a display apparatus. It is a hardware block diagram of an additional information management system and an image management system. FIG. 6 is a diagram showing the structure of a compressed data file in an image file format standard for digital still cameras. FIG. 2 is a functional block diagram of a display device, an additional information management system, and an image management system according to the embodiment. It is a conceptual diagram which shows an additional information management table. It is a conceptual diagram which shows an image management table. It is a sequence diagram showing processing of uploading a photographed image. It is the figure which showed the image object. It is a sequence diagram showing processing of download of a photography picture. It is the flowchart which showed the search processing of additional information. It is the figure which showed the picked-up image in the case of not performing image correction by 2D. It is the figure which showed the picked-up image in the case of not performing image correction by 3D. FIG. 10 is a sequence diagram showing processing in the case where the captured image can not be downloaded.

  Hereinafter, embodiments of the present invention will be described using FIGS. 1 to 24. FIG.

<< Overview of the embodiment >>
First, an outline of the present embodiment will be described with reference to FIGS. 1 to 10. FIG. 1 is a schematic view of an image sharing system according to the present embodiment.

  As shown in FIG. 1, the image sharing system according to the present embodiment is constructed by the imaging device 1, a plurality of display devices (3 a and 3 b), the additional information management system 5, and the image management system 7. The display devices (3a, 3b) are used by the users (A, B), respectively. Further, in the present embodiment, the imaging device 1 is illustrated as being operated by the user A. In addition, below, the arbitrary display apparatuses are represented as the "display apparatus 3" among several display apparatuses (3a, 3b).

  Moreover, the imaging device 1 is a digital camera for obtaining an omnidirectional panoramic image. The display device 3 is a computer such as a smartphone, a tablet terminal, a notebook computer, a desktop computer, or a PDA (Personal Data Assistance). Furthermore, the additional information management system 5 and the image management system 7 are respectively single or plural server computers.

  Further, the imaging device 1 can communicate with the display device 3 by a short distance wireless technology according to the NFC (Near Field Communication) standard. Furthermore, the display device 3 can communicate with the additional information management system 5 and the image management system 7 via the communication network 9. The communication network 9 is constructed by a wireless communication network by a mobile communication system such as 3G (3rd Generation) or WiMAX (Worldwide Interoperability for Microwave Access), and each base station (9a, 9b, 9c), and the Internet. Note that wired communication may be performed between the imaging device 1 and the display device 3 and between the display device 3 and the communication network 9, respectively.

  Next, the appearance of the photographing device 1 will be described with reference to FIG. 2 (a) is a left side view of the imaging apparatus, FIG. 2 (b) is a front view of the imaging apparatus, and FIG. 2 (c) is a plan view of the imaging apparatus.

  As shown in FIG. 2A, the photographing apparatus 1 has a size that can be held by a human with one hand. Further, as shown in FIGS. 2A, 2B, and 2C, the image pickup device 103a is on the front side (front side) at the top of the imaging apparatus 1, and the image pickup device is on the back side (rear side). 103 b is provided. Further, as shown in FIG. 2B, an operation unit 115 such as a shutter button is provided on the front side of the photographing device 1.

  Next, the use condition of the imaging device 1 will be described with reference to FIG. FIG. 3 is an image diagram showing the use situation of the photographing apparatus. The photographing device 1 is used by the user to hold a hand and photograph an object around the user, as shown in FIG. In this case, two hemispherical images can be obtained by imaging the subject around the user with the imaging element 103a and the imaging element 103b shown in FIG.

  Next, an image photographed by the photographing device 1 and a synthesized image will be described with reference to FIG. 4 (a) is a hemispherical image (front side) captured by the imaging device, FIG. 4 (b) is a hemispheric image (back side) captured by the imaging device, and FIG. 4 (c) is represented by Mercator projection. FIG. 6 is a diagram showing a captured image (hereinafter referred to as “mercator image”).

  As shown in FIG. 4A, the image obtained by the imaging element 103a is a hemispherical image (front side) curved by a fisheye lens 102a described later. Further, as shown in FIG. 4B, the image obtained by the imaging element 103b is a hemispherical image (rear side) curved by a fisheye lens 102b described later. Then, the hemispherical image (front side) and the hemispherical image inverted 180 degrees (rear side) are synthesized by the photographing device 1, and as shown in FIG. 3C, a Mercator image is created.

  Next, with reference to FIGS. 5 and 6, the relationship between the Mercator image and the sharing selection image used to select this image will be described. FIG. 5 (a) is a view showing a part of a Mercator image and a share selection image, and FIG. 5 (b) is a view showing a share selection image. FIG. 6 is a diagram of a photographed image selection list showing images for sharing selection.

  The sharing selection image of FIG. 5B is created by utilizing the portion indicated by the broken line on the Mercator image of FIG. 5A. The sharing selection image is transmitted from the imaging device 1 shown in FIG. 1 to the display device 3a. Then, on the display 315 of the display device 3a, as shown in FIG. 6, the shared image selection list SL in which each sharing selection image CE is shown is displayed. For example, a building in a commercial area is displayed as the sharing selection image CE, the shooting date and time (September 20, 2011 11:21), the image data size (13.1 MB) of the captured image, and the attention point are set. Number (0) is displayed.

  Next, when the user A selects a desired sharing selection image CE, the display device 3a acquires a captured image (Mercule image) related to the selected sharing selection image CE from the imaging device 1.

  The captured image thus obtained is shown in FIG. 7 from the Mercator image shown in FIG. 4C by using OpenGL ES (Open Graphics Library for Embedded Systems) in the display device 3a. Panoramic image is created. OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. Further, FIG. 7 is a view showing an omnidirectional panoramic image. The panoramic panoramic image may be a still image or a moving image.

  Next, processing of creating and displaying an image of a predetermined area in the panoramic panoramic image (hereinafter, referred to as “predetermined area image”) will be described using FIGS. 8 and 9. FIG. 8 is a diagram showing the positions of a virtual camera and a predetermined area in the case where the omnidirectional panoramic image is a three-dimensional solid sphere. 9 (a) is a perspective view of FIG. 8 and FIG. 9 (b) is a view showing a display device on which a predetermined area image is displayed.

  Assuming that the omnidirectional panoramic image shown in FIG. 7 is a three-dimensional solid sphere CS, the virtual camera IC is located at the center of the omnidirectional panoramic image, as shown in FIG. It is possible to rotate 3 axes of rotation from the center, up and down, left and right, and rotation (inclination) from a viewpoint from this center. The predetermined region T in the omnidirectional panoramic image is specified by the position coordinates (x, y) of the virtual camera IC and the angle of view in the omnidirectional panoramic image. Thereby, the zoom of the predetermined area T can be expressed by expanding or contracting the range (arc) of the angle of view.

  Then, as shown in FIG. 9 (a), the image of the predetermined area T in the omnidirectional panoramic image is predetermined on the display 315 of the display device 3a as shown in FIG. 9 (b). It is displayed as an area image. The predetermined area image in this case is a partial image P showing a part of the omnidirectional panoramic image. On the display 315, a predetermined area image display area 3110 for displaying a predetermined area image, a thumbnail display area 3120 for displaying thumbnails, and a comment display area 3130 for displaying a user's comment on the predetermined area image are displayed.

  Furthermore, the photographed image shown in FIG. 4C is uploaded from the display device 3a as a transmission device to the image management system 7 via the communication network 9. Then, for example, the user B downloads the captured image from the image management system 7 via the communication network 9 using the display device 3 b as a receiving device. Thereby, each user can share the photographed image.

  Further, as shown in FIG. 10, in the image sharing system of the present embodiment, it is possible to set an attention point at which each user is focusing on the photographed image. FIG. 10 (a) is a view showing a predetermined area image in which each attention point is shown, and FIG. 10 (b) is a view showing an arbitrary attention point and a predetermined area image around it.

  As shown in FIG. 10A, on the display 315 of the display device 3b, a plurality of attention points set by attention by one or more users using the image sharing system of the present embodiment. (AP1, AP2, AP3, AP4) is displayed. Furthermore, an image of a predetermined area T in a certain range including each attention point (AP1, AP2, AP3, AP4) is displayed in the state of thumbnails (T1, T2, T3, T4) in the lower part of the display 315, respectively. . Then, when the user B selects, for example, the thumbnail T1, as shown in FIG. 10B, the predetermined area image (here, the partial image P1), which is the original image of the thumbnail T1, is displayed on the display 315 Will be displayed on.

  Moreover, it is also possible to further set an attention point to the predetermined area image shown in FIG. 10 (b). In this case, a thumbnail relating to the predetermined area image including the set attention point is displayed.

  In the above, the display device 3a is described as the transmission device and the display device 3b is described as the reception device. However, the present invention is not limited to this. The display device 3a may be used as the reception device and the display device 3b as the transmission device. That is, the photographed image can also be downloaded from the image management system 7 also in the display device 3a, and setting of the attention point as shown in FIG. 10 or an image as shown in FIG. It can be done.

<< Hardware Configuration of Embodiment >>
Next, the hardware configurations of the imaging device, the display device, the additional information management system, and the image management system according to the present embodiment will be described in detail with reference to FIGS.

  First, the hardware configuration of the photographing device 1 will be described with reference to FIG. FIG. 11 is a hardware configuration diagram of the photographing apparatus. Hereinafter, the imaging device 1 is an omnidirectional imaging device using two imaging elements, but the number of imaging elements may be three or more. Also, the device is not necessarily a device dedicated to omnidirectional imaging, and a retrofit omnidirectional imaging unit may be attached to an ordinary digital camera, a smartphone or the like to have substantially the same function as the imaging device 1 .

  As illustrated in FIG. 11, the imaging apparatus 1 includes an imaging unit 101, an image processing unit 104, an imaging control unit 105, a central processing unit (CPU) 111, a read only memory (ROM) 112, and a static random access memory (SRAM). A memory 113, a dynamic random access memory (DRAM) 114, an operation unit 115, a network I / F 116, a communication unit 117, and an antenna 117a.

  Among them, the imaging unit 101 includes two wide-angle lenses (so-called fisheye lenses) 102 a and 102 b each having an angle of view of 180 ° or more for imaging a hemispherical image, and two imaging provided corresponding to each wide-angle lens Elements 103a and 103b are provided. The image sensors 103a and 103b convert an optical image by a fisheye lens into image data of an electrical signal and output the image data such as a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor, horizontal or horizontal image sensor The circuit includes a timing generation circuit that generates a vertical synchronization signal, a pixel clock, and the like, and a register group in which various commands and parameters necessary for the operation of the imaging device are set.

  The imaging elements 103a and 103b of the imaging unit 101 are connected to the image processing unit 104 by a parallel I / F bus, respectively. On the other hand, the imaging elements 103 a and 103 b of the imaging unit 101 are connected separately from the imaging control unit 105 by a serial I / F bus (such as an I2C bus). The image processing unit 104 and the imaging control unit 105 are connected to the CPU 111 via the bus 110. Further, to the bus 100, the ROM 112, the SRAM 113, the DRAM 114, the operation unit 115, the network I / F 116, the communication unit 117, the electronic compass 118, and the like are also connected.

  The image processing unit 104 takes in the image data output from the imaging elements 103a and 103b through the parallel I / F bus, performs predetermined processing on each of the image data, and then combines these image data. , Create a Mercator image data as shown in FIG. 4 (c).

  In general, the imaging control unit 105 sets a command or the like in the register group of the imaging elements 103a and 103b using the imaging control unit 105 as a master device and the imaging elements 103a and 103b as slave devices and using an I2C bus. Necessary commands and the like are received from the CPU 111. The imaging control unit 105 also takes in status data and the like of the register group of the imaging elements 103 a and 103 b using the I 2 C bus, and sends the status data to the CPU 111.

  The imaging control unit 105 also instructs the imaging elements 103a and 103b to output image data at the timing when the shutter button of the operation unit 115 is pressed. Depending on the photographing apparatus, there may be a function corresponding to a preview display function by a display or a moving image display. In this case, output of image data from the imaging elements 103a and 103b is continuously performed at a predetermined frame rate (frame / minute).

  Further, as described later, the imaging control unit 105 also functions as a synchronization control unit that synchronizes the output timing of the image data of the imaging elements 103a and 103b in cooperation with the CPU 111. In the present embodiment, the display unit is not provided in the imaging device, but a display unit may be provided.

  The CPU 111 controls the entire operation of the photographing apparatus 1 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. An SRAM 113 and a DRAM 114 are work memories, and store programs executed by the CPU 111, data during processing, and the like. In particular, the DRAM 114 stores image data being processed by the image processing unit 104 and data of the processed Mercator image.

  The operation unit 115 is a generic term such as various operation buttons, a power switch, a shutter button, and a touch panel having both display and operation functions. The user operates the operation buttons to input various shooting modes, shooting conditions, and the like.

  The network I / F 116 is a generic term for an external medium such as an SD card or an interface circuit (USB I / F or the like) with a personal computer or the like. The network I / F 116 may be a network interface regardless of whether it is wireless or wired. The data of the Mercator image stored in the DRAM 114 is recorded on an external medium via the network I / F 116 or, if necessary, the display device 3 via the network I / F 116 serving as the network I / F. Etc. are sent to the external device.

  The communication unit 117 communicates with an external device such as the display device 3 through a short distance wireless technology such as WiFi (wireless fidelity) or NFC via an antenna 117 a provided in the imaging device 1. The communication unit 117 can also transmit the data of Mercator image to the external device of the display device 3.

  The electronic compass 118 calculates the azimuth and tilt (Roll rotation angle) of the imaging device 1 from the magnetism of the earth, and outputs azimuth / tilt information. The azimuth / tilt information is an example of metadata along Exif, and is used for image processing such as image correction of a photographed image. The following description will be made using “additional information” as a broad concept including metadata.

  Next, the hardware configuration of the display device 3 will be described with reference to FIG. FIG. 12 is a hardware configuration diagram when the display device is a smartphone.

  As shown in FIG. 12, the display device 3 includes a CPU 301 that controls the overall operation of the display device 3, a ROM 302 that stores a basic input / output program, and a random access memory (RAM) 303 used as a work area of the CPU 301. Electrically erasable and programmable ROM (EEPROM) 304 that reads or writes data according to control of CPU 301, CMOS sensor 305 as an imaging element for capturing an object and obtaining image data according to control of CPU 301, electronic magnetic compass for detecting geomagnetism And a media drive 308 for controlling reading or writing (storage) of data with respect to a recording medium 307 such as a flash memory or a variety of acceleration / orientation sensors 306 such as a gyro compass and an acceleration sensor. Then, according to the control of the media drive 308, the recording medium 307 on which the data already recorded is read or data is newly written and stored is removable.

  Note that the EEPROM 304 stores an operating system (OS) executed by the CPU 301, other programs, and various data. Also, instead of the CMOS sensor 305, a CCD sensor may be used.

  Furthermore, the display device 3 includes a voice input unit 311 for converting voice into a voice signal, a voice output unit 312 for converting a voice signal into voice, an antenna 313a, and a wireless communication signal using the antenna 313a. Unit 313 for communicating with the stations 9a etc., GPS signals including position information (latitude, longitude, and altitude) of the display 3 by GPS (Global Positioning Systems) satellites or IMES (Indoor MEssaging System) as indoor GPS GPS receiver 314 to receive, a display 315 such as liquid crystal or organic EL, etc. displaying an image of an object or various icons, etc., mounted on the display 315, constituted by a pressure sensitive type or electrostatic type panel, a finger, a touch pen, etc. Touch panel 316 that detects a touch position on the display 315 by a touch by the And it includes an address bus, a data bus, and the like of the bus line 310 for electrically connecting the respective units.

  The voice input unit 311 includes a microphone for inputting voice, and the voice output unit 312 includes a speaker for outputting voice.

  Next, the hardware configuration of the additional information management system 5 and the image management system 7 will be described with reference to FIG. FIG. 13 is a hardware configuration diagram of the additional information management system 5 and the image management system 7. Since both the additional information management system 5 and the image management system 7 are general server computers, the configuration of the additional information management system 5 will be described below, and the description of the configuration of the image management system 7 will be omitted.

  The additional information management system 5 includes a CPU 501 that controls the overall operation of the additional information management system 5, a ROM 502 that stores programs used to drive the CPU 501 such as IPL, a RAM 503 that is used as a work area of the CPU 501, and an additional information management system 5. Data read from or written to an HDD 504 that stores various data such as a program for writing, HDD (Hard Disk Drive) 505 that controls reading or writing of various data to the HD 504 according to the control of the CPU 501, Media drive 507 for controlling storage, a display 508 for displaying various information such as cursors, menus, windows, characters, or images, and a network for performing data communication using the communication network 9 / F 509, a keyboard 511 having a plurality of keys for inputting characters, numerical values, various instructions, etc., selection and execution of various instructions, selection of processing target, mouse 512 for moving a cursor, etc., removable recording medium CD-ROM drive 514 which controls reading and writing of various data to a CD-ROM (Compact Disc Read Only Memory) 513 as one example, and the above respective components are electrically shown as shown in FIG. Bus lines 510 such as an address bus and a data bus for connection are provided.

  FIG. 14 is a diagram showing the structure of a compressed data file in the image file format standard for digital still cameras. The image data obtained by the photographing device 1 has a compressed data file structure, as shown in FIG. This is shown in "Exif 2.3 Image Format Standard for Digital Cameras" enacted on April 26, 2010 (http://www.cipa.jp/hyoujunka/kikaku/pdf/DC-008-2010_J.pdf) Compressed data file structure. Among these, APP1 indicates an application data segment 1 used to record metadata on an image. In this APP1, an EXIF IFD capable of storing unique metadata (Maker Note) for each device maker such as a digital camera is constructed. In the present embodiment, as the additional information, image type information indicating the type of the image, and azimuth / inclination information calculated by the electronic compass 118 are shown. The image type information is, for example, an identification code indicating that the image is an omnidirectional panoramic image, and an identification code indicating that the image is a normal planar image.

<< Functional Configuration of Embodiment >>
Next, a functional configuration of the present embodiment will be described using FIG. FIG. 15 is a functional block diagram of the display device 3, the additional information management system 5, and the image management system 7 which constitute a part of the image sharing system of the present embodiment. In FIG. 15, the display device 3, the additional information management system 5, and the image management system 7 can perform data communication via the communication network 9.

<Functional Configuration of Display Device>
As shown in FIG. 15, the display device 3 includes a transmission / reception unit 31, an operation input reception unit 32, a creation unit 33, a display control unit 34, an image processing unit 38, and a storage and extraction unit 39. . These functions are realized by any one of the components shown in FIG. 11 being operated by an instruction from CPU 111 according to a program for display device 3 expanded on SRAM 114 from DRAM 113 or It is a means.

  Further, the display device 3 includes a storage unit 3000 constructed by the ROM 112, the SRAM 113, and the DRAM 114 shown in FIG.

(Each functional configuration of display device)
Next, each functional configuration of the display device 3 will be described in more detail using FIG. 12 and FIG.

  The transmission / reception unit 31 of the display device 3 is mainly realized by the processing of the communication unit 313 and the CPU 301 shown in FIG. 12, and the additional information management system 5 or the image management system 7 and various data via the communication network 9 Send and receive (or information).

  The operation input receiving unit 32 is mainly realized by the processing by the touch panel 316 and the CPU 301, and receives various selections or inputs from the user.

  The creating unit 33 is mainly realized by the processing of the CPU 301, and based on the designation or selection of the user accepted by the operation input accepting unit 32, from the omnidirectional panoramic image, as shown in FIG. Create an image Also, the creation unit 33 creates the thumbnails shown in FIG. 10 based on the predetermined area image.

  The display control unit 34 is mainly realized by the processing of the CPU 301, and performs control for displaying various images, characters, and the like on the display 315. The display control unit 34 displays, for example, the image data subjected to the image processing by the image processing unit 38 on the display 315 of the display device 3. In addition, the display control unit 34 causes the display 315 to display a predetermined area image related to the thumbnail selected by the operation input receiving unit 32.

  The image processing unit 38 processes the image data using the additional information. This image processing is image editing, image processing, or image correction (correction). The image processing unit 38 performs balloon addition, text addition, trimming, resizing, and the like as image editing. Further, the image processing unit 38 performs image cutting out, sharpening, embossing, blurring, or the like as image processing. Furthermore, the image processing unit 38 performs red-eye correction, brightness adjustment, contrast adjustment, azimuth correction, tilt correction, and the like as image correction.

  The storage / extraction unit 39 stores various data (or information) in the storage unit 3000 or extracts various data (or information) from the storage unit 3000. The storage / extraction unit 39 records, for example, each data of a photographed image, an image for sharing selection, or a predetermined area image in the storage unit 3000 or extracts it from the storage unit 3000.

<Functional configuration of additional information management system>
Next, each functional configuration of the additional information management system 5 will be described in detail using FIG. The additional information management system 5 includes a transmission / reception unit 51, a determination unit 52, an acquisition unit 55, and a storage / extraction unit 59. These units are realized by any one of the components shown in FIG. 13 being operated by an instruction from the CPU 501 according to a program for the additional information management system 5 developed on the RAM 503 from the HD 504. Function or means.

  The additional information management system 5 also has a storage unit 5000 constructed by the RAM 503 and the HD 504 shown in FIG. In the storage unit 5000, an additional information management DB 5001 configured of an additional information management table described later is constructed.

(Additional information management table)
FIG. 16 is a conceptual diagram showing the additional information management table. In the additional information management table, user identification information for identifying a user, image identification information for identifying image data, and the above-described additional information are stored and managed in association with each other.

  Although FIG. 16 shows a user ID (Identification) as an example of the user identification information, it is not necessary to be a user ID if the user can be identified. For example, the user identification information includes a user name, an employee number, a student ID number, a national number based on a national gross number system, and the like. Further, in FIG. 16, an image ID is shown as an example of the image identification information, but if image data can be specified, it is not necessary to be an image ID. For example, the image identification information also includes the file name of the image data.

(Each functional configuration of additional information management system)
Next, each functional configuration of the additional information management system 5 will be described in detail using FIG. 13 and FIG.

  The transmission / reception unit 51 of the additional information management system 5 is mainly realized by the processing of the network I / F 509 and the CPU 501 shown in FIG. 13, and via the communication network 9 the display device 3 or the image management system 7 Send and receive data (or information).

  Determination unit 52 is mainly implemented by CPU 501 shown in FIG. 13 and determines whether additional information corresponding to the image identification information transmitted from display device 3 is stored in the additional information management table or not. to decide.

  The acquisition unit 55 acquires, from the image data stored in the storage unit 5000 after being received by the transmission / reception unit 51, the additional information added to the image data. Note that the additional information may be added to the original image data as a result of this acquisition, or even if the additional information is not added to the original image data (the separated state). Good.

  The storage and extraction unit 59 stores various data (or information) in the storage unit 5000, and extracts various data (or information) from the storage unit 5000.

<Functional Configuration of Image Management System>
Next, the functional configuration of the image management system 7 will be described in detail using FIGS. 13 and 15. The image management system 7 includes a transmission / reception unit 71, a creation unit 72, and a storage / extraction unit 79. These functions are realized by any one of the components shown in FIG. 13 being operated by an instruction from the CPU 501 according to a program for the image management system 7 developed on the RAM 503 from the HD 504. Or means.

  Further, the image management system 7 has a storage unit 7000 constructed by the RAM 503 and the HD 504 shown in FIG. In the storage unit 7000, an image management DB 7001 configured of an image management table described later is constructed.

(Image management table)
FIG. 17 is a conceptual diagram showing an image management table. In this image management table, data of image objects are managed. The image object indicates user identification information for identifying a user, image identification information for identifying an image, and a URL (Uniform Resource Locator) which is a storage location in the storage unit 7000 of image data related to the image identification information. URL information and attribute information are managed in association with each other.

  The user identification information and the image identification information of the image object are the same as the user identification information and the image identification information shown in FIG. On the other hand, the attribute information is different from the additional information shown in FIG. That is, the attribute information is the essential information (image file name, information indicating the number of vertical and horizontal pixels of the image, etc.) of the image data generated by the generation unit 72 described later based on the image data. It is not used for image processing of data.

(Each functional configuration of the image management system)
Next, each functional configuration of the image management system 7 will be described in detail using FIG.

  The transmission / reception unit 71 of the image management system 7 is mainly realized by the processing of the network I / F 509 and the CPU 501 shown in FIG. 13, and the display device 3 or the additional information management system 5 and various types via the communication network 9. Send and receive data (or information).

  In order to manage the image data sent from the display device 3 by the image management system 7, the creation unit 72 assigns image identification information to the image data, and assigns the image identification information to the header portion of the image data. Do. Also, the creation unit 72 creates an image object.

  The storage / extraction unit 79 stores various data (or information) in the storage unit 7000 or extracts various data (or information) from the storage unit 5000.

<< Process or Operation of Embodiment >>
Next, processing of uploading image data by the user A using the display device 3a will be described using FIG. 18 and FIG. FIG. 18 is a sequence diagram showing a process of uploading a photographed image. FIG. 19 shows an image object.

  The communication protocol via the communication network 9 among the display device 3a, the additional information management system 5, and the image management system 7 is HTTP communication by hypertext communication protocol (HTTP). The display device 3a corresponds to an HTTP client, and the additional information management system 5 and the image management system 7 correspond to an HTTP server.

  Unless otherwise noted, arrows from the display device 3a to the additional information management system 5 and the image management system 7 in each sequence diagram correspond to HTTP requests, and from the additional information management system 5 and the image management system 7 to the display device 3a The arrow of corresponds to the HTTP response.

  First, the user A takes in the image data to which the additional information is added from the photographing device 1 to the storage unit 3000 of the display device 3a.

  Next, by selecting the image data to be uploaded by the user A, the operation input receiving unit 32 of the display device 3a receives a selection of the image data to be uploaded (step S1). Then, the transmission / reception unit 31 of the display device 3a transmits the user identification information of the user A, the image data to which the additional information is added, and the data of the file name of this image data to the additional information management system 5 via the communication network 9. To transmit the image data (step S2). Thereby, the transmitting and receiving unit 51 of the additional information management system 5 receives the user identification information and the image data to which the additional information is added.

  Next, the acquisition unit 55 of the additional information management system 5 acquires additional information from the image data downloaded in step S1 (step S3). Then, the storage / extraction unit 59 associates the user identification information received in step S2 with the additional information acquired in step S3 and stores and manages the additional information management table (see FIG. 16) (see FIG. 16). Step S4).

  Next, the transmitting and receiving unit 51 transmits the image data and the user identification information received in step S2 to the image management system 7 via the communication network 9 (step S5). Thus, the transmitting and receiving unit 71 of the image management system 7 receives the image data and the user identification information.

  Next, the creation unit 72 of the image management system 7 creates an image object as shown in FIG. 19 based on the image data received in step S5 (step S6). The data of the image object includes user identification information, image identification information, URL information, and attribute information.

  Next, the storage and extraction unit 79 stores and manages the image object created at step S6 in the image management table (see FIG. 17) (step S7). Then, the transmitting and receiving unit 71 transmits the image object to the additional information management system 5 via the communication network 9 (step S8). Thereby, the transmitting and receiving unit 51 of the additional information management system 5 receives the image object.

  The storage / extraction unit 59 of the additional information management system 5 stores the image identification information of the image object received at step S8 in association with the user identification information and the additional information stored at step S4. Manage by (step S9). Thereby, the additional information management table as shown in FIG. 16 is constructed.

  Then, the transmission / reception unit 51 transmits the image identification information together with the result indicating that the management of the additional information is completed to the display device 3a via the communication network 9 (step S10). Thus, the process of uploading the image data is completed.

  Next, processing of downloading image data by the user B using the display device 3b will be described using FIG. 20 to FIG. FIG. 20 is a sequence diagram showing download processing of a photographed image.

  First, the user B requests the image management system 7 from the transmission / reception unit 31 of the display device 3b via the communication network 9 for a file name list indicating the file names of the images to be downloaded (step S21). Thereby, the transmitting and receiving unit 71 of the image management system 7 receives the request for the file name list.

  Further, in response to this, the storage and extraction unit 79 of the image management system 7 reads the file name information of the image identification information and the additional information from the image management table (see FIG. 17) (step S22). Then, the transmission / reception unit 71 transmits data of a file name list including the image identification information and the file name information corresponding to the image identification information to the display device 3b via the communication network 9 (step S23). Thereby, the transmission / reception unit 31 of the display device 3b receives the data of the file name list.

  Next, when the user B selects a file name indicating the image data to be downloaded from the file name list, the operation input receiving unit 32 of the display device 3b receives the selection of the file name (step S24). Then, the transmission / reception unit 31 of the display device 3b transmits, to the additional information management system 5 via the communication network 9, image identification information corresponding to the file name accepted in step S24 and requests additional information (step S25). As a result, the transmission / reception unit 51 of the additional information management system 5 receives the image identification information and receives a request for additional information.

  Next, the additional information management system 5 performs a search process for additional information (step S26). This search process will be described with reference to FIG. FIG. 21 is a flowchart showing a search process of additional information.

  First, as shown in FIG. 21, the storage / extraction unit 59 searches the additional information management table (see FIG. 16) using the image identification information received in step S25 as a search key (step S101). .

  Next, determination unit 52 determines whether or not corresponding additional information is stored in step S101 (step S102). When it is determined by the determination unit 52 that the corresponding additional information is stored (YES), the storage and extraction unit 59 reads the corresponding additional information, and the process of step S27 in FIG. 20 is subsequently performed. Is done. On the other hand, if the determination unit 52 determines that the corresponding additional information is not stored (NO), the process of step S41 in FIG. 24 described later is performed.

  Next, if it is determined in step S102 that the corresponding additional information is stored (YES), the description will be continued from the process of step S27 of FIG. The transmission / reception unit 51 transmits the additional information to the display device 3b via the communication network 9, as shown in FIG. 20 (step S27). Thereby, the transmission / reception unit 31 of the display device 3b receives the additional information.

  Next, the display device 3b transmits the image identification information of the image that the user B wants to download to the image management system 7 via the communication network 9 and requests the image object (step S28). As a result, the transmission / reception unit 71 of the image management system 7 receives the image identification information and receives the request for the image object.

  Next, the storage and extraction unit 79 of the image management system includes the image identification information by searching the image management table (see FIG. 17) using the image identification information received in step S28 as a search key. An image object is extracted (step S29). Then, the transmitting and receiving unit 71 transmits data of the image object extracted at step S29 to the display device 3b via the communication network 9 (step S30). Thereby, the transmitting and receiving unit 31 of the display device 3b receives the data of the image object.

  Next, the transmission / reception unit 31 requests the image management system 7 for image data managed by the URL by accessing the URL included in the image object (step S31). On the other hand, the storage and extraction unit 79 of the image management system 7 extracts the image data requested from the storage unit 7000 (step S32). Then, the transmitting and receiving unit 79 transmits the image data extracted in step S32 to the display device 3b via the communication network 9 (step S33). Thereby, the transmission / reception unit 31 of the display device 3b receives the image data.

  Next, the image processing unit 38 of the display device 3b performs image processing on the image data received in step S33 based on the additional information received in step S27 (step S34). Then, the display device 3b causes the display 315 to display the image data after the image processing (step S35).

  Here, image correction for correcting a horizon as an example of image processing will be described with reference to FIGS. 22 and 23. FIG. 22 is a diagram showing a photographed image in the case of not performing image correction in 2D. FIG. 23 is a diagram showing a photographed image in the case where the image correction is not performed in 3D.

  For example, when the user A shoots an object with the imaging device 1, if the imaging device 1 is tilted with imaging taken, the image correction is not performed by the display device 3b, as shown in FIG. 22 (in the case of 2D). Alternatively, the inclined image remains as shown in FIG. 23 (when displayed in 3D). However, when the display control unit 34 visualizes the 2D image data shown in FIG. 22 into 3D, the image processing unit 38 corrects the display position based on the tilt information of the azimuth / tilt information. As a result, as shown in FIG. 7, the display control unit 34 can cause the display 315 to display a panoramic panoramic image in which the horizon is horizontal. After that, when the user B touches the display 315, the operation input reception unit 32 can receive a touch, and the display control unit 34 can display a predetermined area image as shown in FIG. 9B.

  Next, processing after it is determined in step S102 that the additional information is not stored will be described using FIG. FIG. 24 is a sequence diagram showing processing in the case where the captured image can not be downloaded.

  As shown in FIG. 24, the transmission / reception unit 51 of the additional information management system 5 transmits an error message indicating that the corresponding additional information is not stored to the display device 3b via the communication network 9. Thereby, the transmission / reception unit 31 of the display device 3b receives the error message.

  Next, the display control unit 34 of the display device 3b displays an error message on the display 315 (step S42). As described above, when the additional information management system 5 does not manage the additional information, the download process can be omitted by preventing the process from proceeding to the process of downloading the image data (steps S31 to S33). . As a use case where this error message is displayed, it can be considered that the image is a normal 2D image which is not a Mercator image. When the captured image is a Mercator image, even if it is displayed as a panoramic panoramic image visualized in 3D as shown in FIG. 7, it is displayed without discomfort, but a normal 2D image which is not a Mercator image is converted to 3D If it is visualized and displayed, it will become more and more curved and it will become a very difficult-to-see display. Therefore, in the present embodiment, if the additional information contributing to the image correction is not stored, the process of downloading the image data is not advanced.

  Note that even if the additional information is not stored, the process of downloading the image data may be performed. In this case, the display device 3 may suppress visualization and display in 3D, and perform normal 2D display.

<< Main effects of this embodiment >>
As described above, according to the present embodiment, the additional information management system 5 acquires additional information from the image data transmitted from the display device 3a (transmission device), while the image identification from the image management system 7 is performed. Receive information The additional information management system 5 can manage the additional information on behalf of the image management system 7 by storing the additional information and the image identification information in association with each other. Thereby, when the user B downloads image data from the image management system 7 by the display device 3 b (receiving device), additional information can be downloaded from the additional information management system 5 based on the corresponding image identification information. The image processing such as correction of image data can be performed.

[Supplement of Embodiment]
The additional information management system 5 and the image management system 7 in each of the above embodiments may be constructed by a single computer, or a plurality of units (functions, means, or storage units) divided and arbitrarily assigned. May be built by a computer.

  In addition, recording media such as CD-ROMs storing the respective programs of the above embodiments and the HD 504 storing these programs may be provided domestically or abroad as a program product (Program Product). it can.

1 Shooting device 3a Display device (an example of a transmitting device)
3b Display (example of receiver)
5 additional information management system 7 image management system 9 communication network 9a, 9b base station 31 transmission / reception unit 32 operation input reception unit 33 creation unit 34 display control unit 38 image processing unit 39 storage / extraction unit 51 transmission / reception unit (an example of transmission means Example of receiving means)
52 Judgment part (an example of judgment means)
55 Acquisition department
59 storage and extraction unit (an example of extraction means)
71. Transmission / reception unit 72. Creation unit (an example of assignment means)
79 storage and extraction unit 315 Display Lee <br/> 3000 memory 5000 storing unit (an example of a second storage means)
5001 Additional Information Management DB
7000 storage unit (an example of a first storage unit)
7001 Image Management DB

JP, 2011-120201, A

Claims (11)

A system capable of communicating with a transmitter that transmits image data to which additional information used for image processing is added, comprising:
Receiving means for receiving the image data from the transmitting device;
First storage means for storing the received image data and image identification information for identifying the image data in association with each other;
A second storage unit that stores the additional information added to the received image data and the image identification information in association with each other;
Transmission means for transmitting the image identification information to the transmission device;
A system characterized by having: The system of claim 1, further comprising:
It has an assigning means for assigning the image identification information to the image data,
A system, wherein the transmitting means transmits the attached image identification information to the transmitting device. The receiving unit receives user identification information for identifying a user of the transmission device, in addition to the image data, from the transmission device.
The system according to claim 1, wherein the second storage unit stores the user identification information in association with the additional information. The receiving unit receives the image identification information from a predetermined receiving device and receives the request for the additional information.
It has an extraction means for extracting corresponding additional information by searching the second storage means based on the image identification information received by the reception means.
The system according to any one of claims 1 to 3, wherein the transmitting means transmits the additional information extracted by the extracting means to the receiving device. The second storage means is searched based on the image identification information received by the reception means to thereby determine whether there is corresponding additional information or not.
When it is determined that the corresponding additional information is present by the determination unit, the transmission unit transmits the corresponding additional information to the reception device, and it is determined that the corresponding additional information is not present by the determination unit 5. A system according to claim 4, wherein the sending means sends an error message to the receiving device.   The system according to any one of claims 1 to 5, wherein the image processing is image editing, image processing, or image correction.   The system according to any one of claims 1 to 6, wherein the image data is data of a Mercator image. A system according to any one of claims 1 to 7;
And the transmission equipment,
An image sharing system characterized by having: A communication method executed by a system capable of communicating with a transmitter that transmits image data to which additional information used for image processing is added,
A receiving step of receiving the image data from the transmitting device;
A first storage step of storing the received image data and image identification information for identifying the image data in association with each other in a first storage unit;
A second storage step of associating the additional information added to the received image data and the image identification information in a second storage unit;
Transmitting the image identification information to the transmitting device;
A communication method characterized by performing.   A program causing a computer to execute the communication method according to claim 9. A system according to claim 4 or 5;
The receiving device;
An image sharing system characterized by having: