WO2020255408A1 - 撮影データ生成装置、撮影データ生成方法及びプログラム - Google Patents

撮影データ生成装置、撮影データ生成方法及びプログラム Download PDF

Info

Publication number
WO2020255408A1
WO2020255408A1 PCT/JP2019/024816 JP2019024816W WO2020255408A1 WO 2020255408 A1 WO2020255408 A1 WO 2020255408A1 JP 2019024816 W JP2019024816 W JP 2019024816W WO 2020255408 A1 WO2020255408 A1 WO 2020255408A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
shooting
data
data generation
captured
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2019/024816
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
敬之 越智
良徳 大橋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Interactive Entertainment Inc
Original Assignee
Sony Interactive Entertainment Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Interactive Entertainment Inc filed Critical Sony Interactive Entertainment Inc
Priority to JP2021528626A priority Critical patent/JP7195430B2/ja
Priority to US17/614,817 priority patent/US12033383B2/en
Priority to PCT/JP2019/024816 priority patent/WO2020255408A1/ja
Publication of WO2020255408A1 publication Critical patent/WO2020255408A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/30—Scenes; Scene-specific elements in albums, collections or shared content, e.g. social network photos or video

Definitions

  • the present invention relates to a shooting data generation device, a shooting data generation method, and a program.
  • the present invention has been made in view of the above problems, and one of the objects thereof is to provide a shooting data generation device, a shooting data generation method, and a program capable of enriching the data accumulated as a life log. ..
  • the shooting data generation device includes a shooting image receiving unit that sequentially receives shot images, an environment map generation unit that generates an environment map based on the plurality of shot images, and the above. It includes a shooting data generation unit that generates shooting data indicating the shooting position of the shot image or the subject of the shot image, which is associated with the environment map.
  • One aspect of the present invention further includes a specific part that specifies a part of the plurality of captured images that satisfies a predetermined condition based on the environmental map and the captured data.
  • the specific unit captures a subject existing at a given position in the environmental map from among a plurality of the captured images based on the environmental map and the captured data. May be specified.
  • the specific unit identifies a part of the plurality of captured images, which is evaluated based on the captured data, based on a certain degree that the captured image associated with the captured data is rare. You may.
  • the specific unit may specify a part of the plurality of captured images based on the density of the position where the captured image is captured.
  • the shooting data may be data indicating the shooting position and shooting direction of the shot image on the environment map, or data indicating the position of the subject on the environment map.
  • the shooting data generation unit may further generate a new image having a higher quality than the shooting image for the specified shooting image.
  • the shooting data generation method includes a step of sequentially accepting shot images, a step of generating an environmental map based on the plurality of shot images, and a shooting position of the shot image associated with the environmental map.
  • it includes a step of generating shooting data indicating the subject of the shot image.
  • the program according to the present invention includes a procedure for sequentially accepting captured images, a procedure for generating an environmental map based on a plurality of the captured images, a photographing position of the captured image or a captured image associated with the environmental map. Have the computer execute the procedure for generating shooting data indicating the subject.
  • FIG. 1 is a configuration diagram showing an example of a life log management system 1 according to an embodiment of the present invention.
  • the life log management system 1 includes a server 10 and a tracker 12.
  • the server 10 and the tracker 12 are connected to a computer network 14 such as the Internet. Then, in the present embodiment, the server 10 and the tracker 12 can communicate with each other.
  • the server 10 is, for example, a server computer used by a user of the life log management system 1.
  • the server 10 includes a processor 20, a storage unit 22, and a communication unit 24.
  • the processor 20 is, for example, a program control device such as a CPU that operates according to a program installed in the server 10.
  • the storage unit 22 is, for example, a storage element such as a ROM or RAM, a hard disk drive, or the like.
  • a program or the like executed by the processor 20 is stored in the storage unit 22.
  • the communication unit 24 is a communication interface such as a network board or a wireless LAN module.
  • the tracker 12 is a device that tracks the position and orientation of the user who wears the tracker 12.
  • the tracker 12 includes a processor 30, a storage unit 32, a communication unit 34, a display unit 36, and a sensor unit 38.
  • the processor 30 is a program control device such as a microprocessor that operates according to a program installed on the tracker 12, for example.
  • the storage unit 32 is, for example, a storage element such as a memory. A program or the like executed by the processor 30 is stored in the storage unit 32.
  • the communication unit 34 is a communication interface such as a wireless LAN module.
  • the display unit 36 is a display arranged in front of the tracker 12, such as a liquid crystal display or an organic EL display.
  • the display unit 36 according to the present embodiment can display a three-dimensional image by displaying, for example, an image for the left eye and an image for the right eye.
  • the display unit 36 may not be able to display a three-dimensional image and may only display a two-dimensional image.
  • the sensor unit 38 is a sensor such as a camera, a microphone, an inertial sensor (IMU), a geomagnetic sensor (direction sensor), a GPS (Global Positioning System) module, a depth sensor, or the like.
  • the camera included in the sensor unit 38 captures an image at a predetermined sampling rate, for example.
  • the microphone included in the sensor unit 38 generates voice data based on the input voice, for example, at a predetermined sampling rate.
  • the inertial sensor included in the sensor unit 38 outputs data indicating the acceleration, rotation amount, movement amount, etc. of the tracker 12 to the processor 30 at a predetermined sampling rate.
  • the geomagnetic sensor included in the sensor unit 38 outputs data indicating the direction in which the tracker 12 faces to the processor 30 at a predetermined sampling rate. Further, the GPS module included in the sensor unit 38 outputs data indicating the latitude and longitude of the tracker 12 to the processor 30 at a predetermined sampling rate.
  • the depth sensor included in the sensor unit 38 is, for example, a depth sensor using technologies such as ToF (Time of Flight), Patterned stereo, and Structured Light.
  • the depth sensor outputs data indicating the distance from the tracker 12 to the processor 30 at a predetermined sampling rate.
  • the sensor unit 38 may include other sensors such as an RF sensor, an ultrasonic sensor, an event driven sensor, a pulse sensor, a heart rate sensor, and a body temperature sensor.
  • sensors such as an RF sensor, an ultrasonic sensor, an event driven sensor, a pulse sensor, a heart rate sensor, and a body temperature sensor.
  • the tracker 12 may include, for example, an HDMI (registered trademark) (High-Definition Multimedia Interface) port, a USB port, an input / output port such as an AUX port, headphones, a speaker, and the like.
  • HDMI registered trademark
  • AUX AUX Port
  • the tracker 12 transmits the sensing data output by the various sensors included in the sensor unit 38 of the tracker 12 to the server 10.
  • SLAM Simultaneous Localization and Mapping
  • self-position estimation for example, at least one of the position and orientation of the tracker 12 is estimated.
  • the global position and orientation of the tracker 12 may be estimated.
  • the SLAM process generates an environmental map based on the sensing data acquired by the tracker 12.
  • the environment map is data showing objects such as point clouds, 3D meshes, and textures, which are generated based on sensing data by, for example, SLAM processing.
  • the above-mentioned sensing data and the environmental map, and various data generated from the sensing data and the environmental map are accumulated in the server 10 as a life log.
  • the accumulated life log data is enriched.
  • FIG. 3 is a functional block diagram showing an example of the functions implemented by the server 10 and the tracker 12 according to the present embodiment. It is not necessary that all the functions shown in FIG. 3 are implemented in the server 10 and the tracker 12 according to the present embodiment, and functions other than the functions shown in FIG. 3 may be implemented.
  • the server 10 functionally includes, for example, a sensing data receiving unit 40, a life log data generating unit 42, and a life log data storage unit 44.
  • the sensing data receiving unit 40 mainly implements the communication unit 24.
  • the life log data generation unit 42 mainly implements the processor 20.
  • the life log data storage unit 44 mainly implements the storage unit 22.
  • the above functions may be implemented by executing a program installed on the server 10 which is a computer and including a command corresponding to the above functions on the processor 20.
  • This program may be supplied to the server 10 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet or the like.
  • the tracker 12 functionally includes, for example, a sensing data acquisition unit 46 and a sensing data transmission unit 48.
  • the sensing data acquisition unit 46 mainly mounts the processor 30 and the sensor unit 38.
  • the sensing data transmission unit 48 is mainly mounted with the communication unit 34.
  • the above functions may be implemented by executing a program installed on the tracker 12 which is a computer and including a command corresponding to the above functions on the processor 30.
  • This program may be supplied to the tracker 12 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet or the like.
  • the sensing data acquisition unit 46 sequentially acquires the sensing data generated by the sensor unit 38 of the tracker 12, for example.
  • the sensing data acquired by the sensing data acquisition unit 46 may include, for example, a captured image captured by the camera included in the sensor unit 38.
  • the camera may capture a moving image.
  • the sensing data acquired by the sensing data acquisition unit 46 may include, for example, a captured image which is a frame of a moving image captured by the camera.
  • the sensing data acquired by the tracker 12 may include voice data generated by the microphone included in the sensor unit 38. Depth data measured by a camera or depth sensor included in the sensor unit 38 of the tracker 12 may be included.
  • the sensing data acquired by the tracker 12 may include data indicating the orientation of the tracker 12 measured by the geomagnetic sensor included in the sensor unit 38 of the tracker 12. Further, the sensing data acquired by the tracker 12 may include data indicating the acceleration, rotation amount, movement amount, etc. of the tracker 12 measured by the inertial sensor included in the sensor unit 38.
  • the sensing data acquired by the tracker 12 may include data indicating the latitude and longitude of the tracker 12 measured by the GPS module included in the sensor unit 38. Further, the sensing data acquired by the tracker 12 may include a feature point cloud (keyframe).
  • the sensing data acquired by the tracker 12 may include health care data indicating pulse, heart rate, body temperature, and the like.
  • the sensing data transmission unit 48 sequentially transmits the sensing data acquired by the sensing data acquisition unit 46 to the server 10.
  • sensing data associated with sensing time point data indicating a time point at which sensing by the tracker 12 is performed is transmitted.
  • the acquisition and transmission of sensing data in the tracker 12 are repeatedly executed.
  • the acquisition and transmission of the sensing data in the tracker 12 may be repeatedly executed at predetermined time intervals.
  • the sensing data receiving unit 40 sequentially receives the sensing data transmitted from the sensing data transmitting unit 48 of the tracker 12, for example.
  • the life log data generation unit 42 generates life log data based on the sensing data received by the sensing data receiving unit 40, for example. Then, in the present embodiment, the life log data generation unit 42 stores the generated life log data in the life log data storage unit 44, for example.
  • the life log data storage unit 44 stores, for example, the above-mentioned life log data.
  • FIG. 4 is a diagram showing an example of the data structure of the life log data stored in the life log data storage unit 44.
  • the life log data includes, for example, an environmental map and a plurality of individual data.
  • the life log data generation unit 42 executes SLAM processing including estimation of the position or orientation of the tracker 12 based on a plurality of sensing data received by the sensing data receiving unit 40, for example.
  • the global position and orientation of the tracker 12 may be estimated.
  • the life log data generation unit 42 may execute SLAM processing including relocalization processing, loop closing processing, 3D meshing processing, object recognition processing, and the like.
  • the SLAM process may include a plane detection / 3D mesh segmentation process.
  • Plane detection / 3D mesh segmentation processing refers to the processing of detecting continuous planes such as the ground and walls and dividing the entire 3D mesh into individual 3D meshes such as the ground, buildings, and trees.
  • the SLAM process may include a 3D mesh optimization process.
  • the 3D mesh optimization process refers to a process of removing presumed moving objects, dust due to noise, etc., reducing the number of polygons, and smoothing the surface of the mesh from the 3D mesh.
  • the SLAM process may include a texture generation process.
  • the texture generation process refers to a process of generating a texture image for a 3D mesh based on the colors of the vertices of the mesh.
  • the life log data generation unit 42 may execute the SLAM process using the time-series sensing data.
  • An environmental map is generated by the above-mentioned SLAM processing executed by the life log data generation unit 42.
  • the life log data generation unit 42 may generate a four-dimensional environment map including a three-dimensional space and time in which a three-dimensional environment map at a designated time can be specified.
  • the environment map generated in this way is stored in the life log data storage unit 44 as a part of the life log data.
  • the life log data generation unit 42 generates, for example, the sensing data received by the sensing data receiving unit 40 and individual data including the sensing time point data associated with the sensing data.
  • FIG. 5 is a diagram showing an example of a data structure of individual data.
  • the ID is identification information of individual data.
  • the individual data includes the sensing data including the captured image received by the sensing data receiving unit 40 and the sensing time point data associated with the sensing data.
  • the life log data generation unit 42 generates the generated life log data based on at least one of the environment map and the sensing data.
  • the generated life log data generated in this way is added to the individual data including the sensing data.
  • the life log data generation unit 42 indicates the shooting position of the shot image or the subject of the shot image associated with the environment map based on the shot image included in the sensing data and the environment map. To generate. Then, the life log data generation unit 42 may add the generated life log data including the shooting data generated in this way to the individual data.
  • the shooting data may be data indicating the shooting position of the shot image and the shooting direction of the shot image.
  • the shooting position and shooting direction are represented by the position and direction in the environment map.
  • the shooting data may be data indicating the shooting position of the shot image and the subject position which is the position of the subject of the shot image.
  • the shooting position and the subject position are represented by positions in the environment map.
  • the generated life log data includes posture data indicating the posture of the whole body of the user wearing the tracker 12, data indicating a person extracted from the captured image, text extracted from the captured image and audio data, and the like. You may.
  • reorganization processing such as deletion of captured images included in life log data and improvement of image quality is executed.
  • the life log data generation unit 42 executes preprocessing (S101).
  • the life log data generation unit 42 deletes, for example, a captured image satisfying a predetermined condition indicating that it is a bad image from a series of captured images included in the life log data. For example, a captured image that satisfies conditions such as blurring or blurring, pure white or pure black is deleted.
  • the life log data generation unit 42 identifies a captured image in which a specific subject such as a building is only partially captured. Then, the life log data generation unit 42 changes the specified photographed image into a photographed image in which the entire subject is captured by combining the photographed image with another frame.
  • the life log data generation unit 42 identifies a plurality of representative images from the captured images stored in the life log data storage unit 44 (S102).
  • the life log data generation unit 42 identifies a plurality of captured images similar to each other, and identifies any one of the plurality of captured images as a representative image corresponding to the plurality of captured images. To do.
  • the life log data generation unit 42 specifies the importance of the representative image for each representative image (S103).
  • the higher the importance the higher the importance of the value is specified.
  • the representative image when the voice data corresponding to the representative image shows an important keyword or a voice representing the name of the above-mentioned predetermined person, the representative image is specified to have a higher importance than the non-representative image. May be good.
  • a representative image showing a subject that the user is gazing at may be specified with a higher importance than an image that does not.
  • a representative image in which a subject that is longer than a predetermined time that the user is gazing at may be specified as having a higher importance than that that is not.
  • the gaze position of the user may be specified based on, for example, the posture of the tracker 12 specified based on the sensing data.
  • the sensor unit 38 includes an eye tracker
  • the gaze position may be specified based on the result of detecting the line of sight by the eye tracker.
  • a place registered as a favorite place by the user or another user a representative image of a predetermined event, a place where the frequency of visits exceeds a predetermined number of times, or a representative image of an event occurring at the place is not so. Higher importance than one may be identified. Further, for a representative image similar to a representative image whose importance value specified in the past is larger than a predetermined value, a higher importance may be specified than that of a representative image that is not.
  • the degree of excitement of the situation appearing in the representative image may be specified. Then, for a representative image having a high degree of excitement, a higher importance may be specified than that of a representative image that does not.
  • a representative image showing a predetermined subject such as a famous landmark may be specified with a higher importance than the other image.
  • the representative image shows a subject at a predetermined location. For example, it may be determined whether or not the representative image shows a subject at a predetermined location based on the environment map, the shooting position, and the shooting direction. Further, for example, it may be determined whether or not the representative image shows the subject at a predetermined location based on the environment map and the subject position indicated by the shooting data.
  • a machine learning model such as deep learning or composition analysis may be used to determine whether or not the representative image represents a good scene. For example, a scene close to one scene of a famous painting or movie may be determined to represent a good scene. And what is determined to be a good scene may be identified as having a higher importance than what is not.
  • the life log data generation unit 42 executes a classification process for classifying a plurality of representative images into image groups associated with each event (S104).
  • the classification process may be executed using the information used for specifying the importance in the process shown in S103 described above.
  • the life log data generation unit 42 satisfies a predetermined condition from the plurality of representative images based on the environment map and the shooting data associated with each of the plurality of representative images.
  • the unit may be specified as an image group associated with one event.
  • the life log data generation unit 42 captures a subject existing at a given position in the environment map from among a plurality of representative images based on the environment map and the shooting data associated with the representative image. You may identify a part of it. Then, the life log data generation unit 42 may classify a part identified in this way into an image group associated with an event for the subject at the position.
  • the representative image For example, based on the environment map and the shooting data associated with the representative image, it may be determined whether or not the representative image shows the subject at a predetermined location. For example, based on the environment map and the shooting position and shooting direction indicated by the shooting data associated with the representative image, it is determined whether or not the representative image shows a subject at a predetermined location. May be good. Further, for example, it may be determined whether or not the representative image shows the subject at a predetermined location based on the environment map and the subject position indicated by the shooting data associated with the representative image. .. Then, a representative image determined to capture a subject at a predetermined location may be classified into an image group associated with an event for the subject.
  • the position of the subject of the photographed image in the environment map can be easily specified based on the environment map and the photographed data associated with the photographed image. Therefore, according to the present embodiment, it is possible to easily identify the captured image with the subject as the key.
  • the classification process for classifying the shooting positions into representative image groups separated from each other may be executed.
  • the classification process may be executed based on the magnitude of the movement amount corresponding to the distance between the shooting positions indicated by the shooting data associated with the representative image. For example, when the distance between the shooting positions indicated by the shooting data associated with each of the two consecutive representative images at the shooting time (sensing time) is longer than a predetermined distance, the image group is between the two representative images. May be split. Similarly, the image group may be divided between two representative images having a large change in acceleration.
  • the image group may be divided between the two representative images.
  • the life log data generation unit 42 identifies at least one important image for each event (S105).
  • the importance corresponding to each event is specified.
  • a representative value such as a total value or an average value of the importance of the representative image corresponding to the event may be specified as the value of the importance corresponding to the event.
  • the higher the importance of the event the larger the number of important images may be identified.
  • a number of representative images corresponding to the event may be specified as important images in order of importance.
  • a predetermined number of important images may be specified in descending order of the product of the importance value of the event and the importance value of the representative image.
  • the life log data generation unit 42 may evaluate to some extent that the representative image is rare based on the shooting data associated with the representative image. Then, the life log data generation unit 42 may specify a part of the plurality of representative images as an important image based on a rare image evaluated in this way.
  • the life log data generation unit 42 may specify a part of the plurality of representative images as an important image based on the density of the positions where the plurality of captured images including the representative image are captured. For example, the life log data generation unit 42 may specify the number of captured images captured in a region having a predetermined size centered on the position where the representative image is captured. Then, the representative image with a small number specified may be preferentially specified as an important image. For example, a representative image whose specified number is smaller than a predetermined number may be specified as an important image.
  • important images may be specified so that the shooting positions are dispersed as much as possible.
  • important images may be specified so that the composition and the shooting time (sensing time) are dispersed as much as possible.
  • an important image may be specified based on health care data such as pulse, heart rate, and body temperature.
  • health care data such as pulse, heart rate, and body temperature.
  • a representative image associated with healthcare data having a large pulse, a large heartbeat, or a high body temperature may be specified as an important image.
  • the life log data generation unit 42 executes post-processing on the important image specified by the process shown in S105 (S106).
  • the life log data generation unit 42 may generate a new image having a higher quality than the important image for the important image.
  • the shooting position and the shooting direction in the environment map may be specified based on the shooting data associated with the important image.
  • the direction from the shooting position indicated by the shooting data toward the subject position indicated by the shooting data may be specified as the shooting direction.
  • an image showing how the environment map is viewed from the shooting position toward the shooting direction may be generated.
  • the life log data generation unit 42 may add the image generated in this way to the individual data including the important image.
  • an image with high image quality of the important image may be generated by using the captured image, the depth information, and the photographing direction for a predetermined number of frames before and after the important image.
  • the depth information may be generated based on, for example, the depth data included in the sensing data. Further, the depth information may be generated based on the environment map and the shooting data. Further, the shooting direction may be specified based on the shooting data. Further, the shooting direction may be specified based on the sensing data. Then, the life log data generation unit 42 may add the image generated in this way to the individual data including the important image.
  • the life log data generation unit 42 may add the image generated in this way to the individual data including the important image.
  • a moving image including these important images may be specified as an important moving image.
  • stabilization processing such as camera shake correction may be executed for the important moving image.
  • the life log data generation unit 42 may add the important moving image on which the stabilization process is executed to the individual data associated with at least one frame among the plurality of frames.
  • the processes shown in S101 to S106 may be executed, for example, at predetermined time intervals.
  • a high-quality image can be obtained for an important scene based on the captured image without separately capturing a high-quality image.
  • the life log according to the present embodiment is, for example, generation of minutes, search of life log based on text, past re-experience by virtual reality (VR), restoration of three-dimensional space from childhood at a specific time, It can be used for conversations by voice synthesis.
  • VR virtual reality
  • the life log according to the present embodiment may include, for example, reproduction of changes at a specific point by fixed point observation, identification of who was with a person at a specific time, places where a person has traveled in the past, and visits. It can be used for good places, extraction of past periodic actions, etc.
  • the user always wears the tracker 12. Then, for the moving image taken by the tracker 12 that the user always wears without being particularly conscious, the above-mentioned preprocessing, identification of the important image, and post-processing for the important image (high image quality, etc.) Is executed. Then, the high-quality still image generated in this way is presented to the user. In this way, even if the user does not consciously take a picture, the user is presented with a high-quality picture of the days spent by the user.
  • the present invention is not limited to the above-described embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Tourism & Hospitality (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Data Mining & Analysis (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Operations Research (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Processing Or Creating Images (AREA)
  • Studio Devices (AREA)
PCT/JP2019/024816 2019-06-21 2019-06-21 撮影データ生成装置、撮影データ生成方法及びプログラム Ceased WO2020255408A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021528626A JP7195430B2 (ja) 2019-06-21 2019-06-21 撮影データ生成装置、撮影データ生成方法及びプログラム
US17/614,817 US12033383B2 (en) 2019-06-21 2019-06-21 Imaging data generation apparatus, imaging data generation method, and program
PCT/JP2019/024816 WO2020255408A1 (ja) 2019-06-21 2019-06-21 撮影データ生成装置、撮影データ生成方法及びプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2019/024816 WO2020255408A1 (ja) 2019-06-21 2019-06-21 撮影データ生成装置、撮影データ生成方法及びプログラム

Publications (1)

Publication Number Publication Date
WO2020255408A1 true WO2020255408A1 (ja) 2020-12-24

Family

ID=74040467

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/024816 Ceased WO2020255408A1 (ja) 2019-06-21 2019-06-21 撮影データ生成装置、撮影データ生成方法及びプログラム

Country Status (3)

Country Link
US (1) US12033383B2 (https=)
JP (1) JP7195430B2 (https=)
WO (1) WO2020255408A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2025048826A (ja) * 2023-09-20 2025-04-03 ソフトバンクグループ株式会社 システム

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011248832A (ja) * 2010-05-31 2011-12-08 Denso It Laboratory Inc 画像収集システム、携帯端末、画像収集装置、及び画像収集方法
JP2018128314A (ja) * 2017-02-07 2018-08-16 富士通株式会社 移動体位置推定システム、移動体位置推定端末装置、情報格納装置、及び移動体位置推定方法

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7627148B2 (en) * 2004-07-06 2009-12-01 Fujifilm Corporation Image data processing apparatus and method, and image data processing program
JP4287465B2 (ja) * 2006-12-18 2009-07-01 シャープ株式会社 画像処理装置
JP4655238B2 (ja) * 2008-09-19 2011-03-23 ソニー株式会社 画像処理装置および方法、並びにプログラム
US9152700B2 (en) * 2011-09-30 2015-10-06 Google Inc. Applying query based image relevance models
US10311096B2 (en) * 2012-03-08 2019-06-04 Google Llc Online image analysis
JP5703255B2 (ja) * 2012-04-27 2015-04-15 株式会社東芝 画像処理装置、画像処理方法およびプログラム
CA2896985A1 (en) * 2013-01-03 2014-07-10 Meta Company Extramissive spatial imaging digital eye glass for virtual or augmediated vision
US20150242405A1 (en) * 2014-02-26 2015-08-27 Sony Corporation Methods, devices and systems for context-sensitive organization of media files
JP6149015B2 (ja) * 2014-09-10 2017-06-14 富士フイルム株式会社 画像処理装置、画像処理方法、プログラムおよび記録媒体
CN105912667A (zh) * 2016-04-12 2016-08-31 玉环看知信息科技有限公司 一种信息推荐方法、装置及移动终端
US10929461B2 (en) * 2016-07-25 2021-02-23 Evernote Corporation Automatic detection and transfer of relevant image data to content collections
US10140675B2 (en) * 2016-11-28 2018-11-27 Google Llc Image grid with selectively prominent images
JP2018110637A (ja) * 2017-01-10 2018-07-19 コニカミノルタ株式会社 動態画像処理装置
JP6905390B2 (ja) * 2017-06-01 2021-07-21 株式会社豊田中央研究所 自車位置推定用環境地図生成装置、自車位置推定装置、自車位置推定用環境地図生成プログラム、及び自車位置推定プログラム
US11170269B2 (en) * 2018-12-12 2021-11-09 International Business Machines Corporation Selective image compression of an image stored on a device based on user preferences

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011248832A (ja) * 2010-05-31 2011-12-08 Denso It Laboratory Inc 画像収集システム、携帯端末、画像収集装置、及び画像収集方法
JP2018128314A (ja) * 2017-02-07 2018-08-16 富士通株式会社 移動体位置推定システム、移動体位置推定端末装置、情報格納装置、及び移動体位置推定方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2025048826A (ja) * 2023-09-20 2025-04-03 ソフトバンクグループ株式会社 システム

Also Published As

Publication number Publication date
JPWO2020255408A1 (https=) 2020-12-24
US12033383B2 (en) 2024-07-09
JP7195430B2 (ja) 2022-12-23
US20220230435A1 (en) 2022-07-21

Similar Documents

Publication Publication Date Title
US11238668B2 (en) Method for detecting objects and localizing a mobile computing device within an augmented reality experience
US11238606B2 (en) Method and system for performing simultaneous localization and mapping using convolutional image transformation
US10488195B2 (en) Curated photogrammetry
US11321385B2 (en) Visualization of image themes based on image content
JP7678189B2 (ja) 拡張現実マップキュレーション
US10482608B2 (en) Method and apparatus to generate haptic feedback from video content analysis
EP4510082A2 (en) Cloud enabled augmented reality
DE112019001257T5 (de) Stabilisierung von video zur verringerung von kamera- und gesichtsbewegung
CN118265996A (zh) 显示图像转换可能性信息的计算设备
CN116193049A (zh) 视频处理方法、装置、计算机设备和存储介质
US20230103650A1 (en) System and method for providing scene information
JP7195430B2 (ja) 撮影データ生成装置、撮影データ生成方法及びプログラム
CN111695424A (zh) 基于三维实景的犯罪现场还原方法、系统和存储介质
CN117119287A (zh) 一种无人机拍摄角度的确定方法、装置及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19933350

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021528626

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19933350

Country of ref document: EP

Kind code of ref document: A1