WO2024201814A1 - 映像共有システム及び映像共有方法 - Google Patents

映像共有システム及び映像共有方法 Download PDF

Info

Publication number
WO2024201814A1
WO2024201814A1 PCT/JP2023/012897 JP2023012897W WO2024201814A1 WO 2024201814 A1 WO2024201814 A1 WO 2024201814A1 JP 2023012897 W JP2023012897 W JP 2023012897W WO 2024201814 A1 WO2024201814 A1 WO 2024201814A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
terminal
photographing
image
reference point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/012897
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
康宣 橋本
眞弓 中出
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Maxell Ltd
Original Assignee
Maxell Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Maxell Ltd filed Critical Maxell Ltd
Priority to CN202380096256.4A priority Critical patent/CN121002887A/zh
Priority to PCT/JP2023/012897 priority patent/WO2024201814A1/ja
Priority to JP2025509427A priority patent/JPWO2024201814A1/ja
Publication of WO2024201814A1 publication Critical patent/WO2024201814A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to a video sharing system and a video sharing method.
  • Patent Document 1 states, "The mount-type device has a camera that captures images of the work site over a range wider than the field of view of the worker, a sensor that acquires the worker's line of sight as sensor information, and a gaze movement detection unit that uses the sensor information to detect the movement of the worker's gaze as gaze movement information, and transmits the captured image data and gaze movement information of the camera to an operation instruction device over a network.
  • the operation instruction device has a display setting unit that cuts out a first image of a range wider than the field of view of the worker from the image captured by the camera, corrects the first image using the gaze movement information, and displays the corrected first image on a display, and an instruction image generation unit that generates an instruction image for the worker, and transmits the instruction image to the mount-type device over the network.
  • the present invention has been made in consideration of the above circumstances, and aims to provide a video sharing system and method that realizes a display that is not affected by the directional movement of the shooting terminal by linking a shooting terminal operated by a person shooting the work situation, etc., with a display terminal that receives and displays the captured video transmitted from the shooting terminal, without using a large fixed display device.
  • the present invention provides a video sharing system configured by connecting a photographing terminal and a display terminal for communication
  • the photographing terminal includes a camera, a first position and orientation sensor that detects the position and orientation of the photographing terminal, a first processor, and a first communication device that transmits and receives data to and from the display terminal
  • the display terminal includes a display, a second position and orientation sensor that detects the position and orientation of the display terminal, a second processor, and a second communication device that transmits and receives data to and from the photographing terminal
  • the first processor of the photographing terminal sets the position of the photographing terminal as a photographing reference point in a photographing reference state set in a real space in which the photographing terminal exists, sets the direction in which the front of the photographing terminal faces from the photographing reference point as a photographing reference direction, transmits the photographing reference point and the photographing reference direction to the display terminal, and based on sensor information of the first position and
  • the second processor of the display terminal acquires the amount of change in the posture of the shadow terminal, and transmits the change in position of the shooting terminal and the amount of change in posture of the shooting terminal as first correction information to the display terminal together with the captured image information generated by the camera capturing the object to be shot.
  • the second processor of the display terminal sets the position of the display terminal as a display reference point and sets the direction from the display reference point to the front of the display terminal as a display reference direction.
  • the second processor acquires the change in position of the display terminal relative to the display reference point and the amount of change in posture of the display terminal relative to the display reference state as second correction information.
  • the first correction information and the second correction information are used to correct the captured image information so that the positional relationship of the object to be shot when the shooting reference point and the shooting reference direction are used as references and the apparent positional relationship of the image of the object to be shot when the display reference point and the display reference direction are used as references coincide with each other.
  • the corrected captured image information is displayed on the display.
  • the present invention provides a video sharing system and method that links a shooting terminal operated by a person shooting the work situation, etc., with a display terminal that receives and displays the captured images sent from the shooting terminal, without using a large fixed display device, and realizes a display that is not affected by the directional movement of the shooting terminal. Objectives, configurations, and effects other than those described above will be made clear in the following embodiments.
  • FIG. 1 is a schematic configuration diagram of a video sharing system according to a first embodiment.
  • FIG. 2 is a hardware configuration diagram of the photographing terminal and the display terminal.
  • FIG. 1 is a hardware configuration diagram of a smartphone.
  • FIG. 2 is a functional block diagram of a photographing terminal 1A portion of the video sharing system.
  • FIG. 2 is a functional block diagram of a display terminal 1B portion of the video sharing system.
  • 1 is a diagram showing an example of a positional relationship between a photography reference point, a photography reference direction, and a photography target object in a photography space in which a photography terminal exists;
  • FIG. 5B is a diagram showing a captured image in the positional relationship of FIG. 5A.
  • FIG. 1A to 1C are diagrams illustrating an example of the positional relationship between a display reference point, a display reference direction, and an image of an object to be photographed in a display space in which a display terminal exists.
  • FIG. 6B is a diagram showing a display image in the positional relationship of FIG. 6A.
  • 5B is a diagram showing an example of a positional relationship when the position and orientation of the photographing terminal are changed from the positional relationship of the position and direction of the photographing terminal shown in FIG. 5A.
  • FIG. 7B is a diagram showing a captured image in the positional relationship of FIG. 7A.
  • 7B is a diagram showing an image on the display terminal when the photographing terminal is in the state of FIG. 7A and the display terminal is in the reference position of FIG. 6A.
  • FIG. 6B is a diagram showing an example of the positional relationship when the position and orientation of the display terminal are changed from the positional relationship of the position and direction of the display terminal shown in FIG. 6A.
  • FIG. 8B is a diagram showing a display image in the positional relationship of FIG. 8A.
  • FIG. 11 is a diagram showing parameters used in the description of an image correction method. 1A and 1B are diagrams showing the relative relationship between a captured image and a displayed image with respect to a reference image.
  • FIG. 11 is an explanatory diagram of parameters used in a real space in which a photographing terminal exists.
  • FIG. 2 is an explanatory diagram of each parameter used in a real space in which a display terminal exists.
  • 4 is a flowchart showing a processing flow of the video sharing system according to the first embodiment.
  • the present invention is expected to improve visibility when supporters providing rear support view images of workers working on-site.
  • the present invention is expected to improve technology for labor-intensive industries that require work support and rear support, and is expected to contribute to 8.2 of the Sustainable Development Goals (SDGs) advocated by the United Nations (increase economic productivity through diversification, technological improvement, and innovation, particularly in industries that increase the value of goods and services and labor-intensive industries).
  • SDGs Sustainable Development Goals
  • FIG. 1 is a schematic diagram of a video sharing system according to the first embodiment.
  • Video information of a subject 7 photographed by a photographing terminal 1A which is a portable information terminal worn by a user 2A
  • a display terminal 1B which is a portable information terminal of a user 2B.
  • information showing the relationship between the video at the time of photographing and the photographing reference point and the photographing reference direction is also transmitted.
  • a subject image 8 is displayed in which the positional relationship of the photographed subject 7 is corrected as if the subject 7 exists at a set position in the space in which the display terminal 1B exists.
  • the communication between the photographing terminal 1A and the display terminal 1B may be via a communication network 4 via access points 3A and 3B, or may be direct communication between the photographing terminal 1A and the display terminal 1B.
  • an HMD Head Mounted Display
  • the imaging terminal 1A and the display terminal 1B are used as the imaging terminal 1A and the display terminal 1B.
  • Users 2A and 2B each operate a smartphone 5A or 5B, and the smartphones 5A and 5B may also be communicatively connected to access points 3A and 3B and the communication network 4.
  • FIG. 2 is a hardware configuration diagram of the photographing terminal and the display terminal. Since photographing terminal 1A and display terminal 1B have the same configuration, in the following explanation, components common to photographing terminal 1A and display terminal 1B are distinguished by adding the suffixes A and B to the reference symbols.
  • the photographing terminal 1A and the display terminal 1B include outer cameras 111A, 111B, distance measurement sensors 113A, 113B, position and orientation sensors 135A, 135B, geomagnetic sensors 117A, 117B, positioning sensors 118A, 118B, RTCs (real-time clocks) 114A, 114B, displays 119A, 119B, microphones 121A, 121B, speakers 122A, 122B, processors 125A, 125B, memories 128A, 128B, and network communication devices 120A, 120B, which are connected to each other via buses 140A, 140B that connect the various components.
  • the network communication devices 120A, 120B are connected to antennas 123A, 123B that transmit and receive network communication signals. Displays 119A and 119B and speakers 122A and 122B correspond to output devices, and microphones 121A and 121B correspond to input devices.
  • the position and orientation sensors 135A and 135B are used to measure the position and orientation of the photographing terminal 1A and the display terminal 1B.
  • orientation means the three-dimensional rotational position of the photographing terminal 1A and the display terminal 1B in the real space in which they exist. Details will be described later.
  • Specific configuration examples of the position and orientation sensors 135A and 135B include position and orientation cameras 136A and 136B (Position and Posture cameras: hereafter abbreviated as "PP cameras”), position and orientation distance measuring sensors 137A and 137B (Position and Posture distance measuring sensors: hereafter abbreviated as PP distance measuring sensors), acceleration sensors 115A and 115B, and gyro sensors 116A and 116B.
  • the PP cameras 136A and 136B may be the same as the outer cameras 111A and 111B.
  • PP distance measuring sensors 137A and 137B may be the same as distance measuring sensors 113A and 113B.
  • Measurement of position and attitude using PP cameras 136A, 136B and PP distance measuring sensors 137A, 137B utilizes feature points that exist in the external world. Multiple feature points are used, and may be, for example, feature points of the object 7 to be photographed, or feature points of an indoor space, or feature points of a building if outdoors. As preparation, the relative positional relationship of the feature points is measured from the direction and distance measurement of the feature points. This preparation can be omitted if the coordinate values of the feature points in the external coordinate system are known. Then, changes in the direction and distance of the feature points as seen from imaging terminal 1A and display terminal 1B are measured to measure changes in the position and attitude of imaging terminal 1A and display terminal 1B.
  • PP cameras 136A, 136B When measuring distance using a stereo camera method, only PP cameras 136A, 136B may be used. Furthermore, if the external coordinate system values of the feature points are known, and the relationship between the external coordinate system and the vertical direction is known, the relative relationship between the housing attitude and the vertical direction of the imaging terminal 1A and the display terminal 1B can be grasped by measuring the attitude using the PP cameras 136A, 136B and the PP distance measuring sensors 137A, 137B.
  • the position and attitude are measured using acceleration sensors 115A, 115B and gyro sensors 116A, 116B by integrating acceleration and angular acceleration. This allows for the determination of the displacement from a reference position and reference attitude.
  • the relative relationship between the body attitude and the vertical direction of the photographing terminal 1A and the display terminal 1B can also be determined by measuring the gravitational acceleration vector using acceleration sensors 115A, 115B.
  • the position and attitude of the photographing terminal 1A and the display terminal 1B may be measured by referring to both the measured values using the PP cameras 136A, 136B and the PP distance measuring sensors 137A, 137B and the measured values using the acceleration sensors 115A, 115B and the gyro sensors 116A, 116B, or by using the measured values of other types of sensors.
  • the positioning sensors 218A, 218B are used to measure the indoor and outdoor positions of the photographing terminal 1A and the display terminal 1B. They are used when searching for library images related to the photographing subject 7. Specific examples of this indoors include those that use beacon signals and those that use position indicator marks. Outdoors, for example, there is a GPS (Global Positioning System) receiver that uses signals from satellites.
  • GPS Global Positioning System
  • the network communication devices 120A, 120B are communication interfaces that perform wireless communication between at least the photographing terminal 1A and the display terminal 1B and the access points 3A, 3B by short-distance wireless communication, wireless LAN, or base station communication, and include communication processing circuits corresponding to various predetermined communication interfaces, and are connected to the antennas 123A, 123B.
  • the network communication devices 120A, 120B transmit and receive biometric information, image data, control signals, and the like.
  • the short-distance wireless communication is performed using wireless LAN such as Bluetooth (registered trademark), IrDA (Infrared Data Association, registered trademark), Zigbee (registered trademark), HomeRF (Home Radio Frequency, registered trademark), or Wi-Fi (registered trademark).
  • long-distance wireless communication such as LTE (Long Term Evolution, registered trademark), LTE-Advanced, Mobile WiMAX (Worldwide Interoperability for Microwave Access, registered trademark), or WiMAX2 may be used.
  • Displays 119A, 119B are placed in front of both eyes of users 2A, 2B wearing the imaging terminal 1A and display terminal 1B.
  • Processors 125A, 125B control the operation of imaging terminal 1A and display terminal 1B. In particular, they correct images of real space information captured by outer cameras 111A, 111B and captured images received from access points 3A, 3B, and display them on displays 119A, 119B.
  • Processors 125A and 125B are configured, for example, by a CPU.
  • Memories 128A and 128B are composed of flash memory and non-volatile memory. Memories 128A and 128B also store programs 126A and 126B such as an OS (Operating System) and operation control applications, as well as data 127A and 127B used by processors 125A and 125B.
  • OS Operating System
  • data 127A and 127B used by processors 125A and 125B.
  • the processors 125A and 125B load the programs 126A and 126B into the memories 128A and 128B and execute them, and read the data 127A and 127B as necessary and use them in the execution processing of the programs 126A and 126B.
  • FIG. 3 shows the hardware configuration of a smartphone.
  • Components with the same names as those of the HMD basically have the same functions, so detailed explanations will be omitted unless there are particular differences.
  • the outer cameras 211A and 211B, the inner cameras 212A and 212B, the distance measuring sensors 213A and 213B, the position and orientation sensors 235A and 235B, the geomagnetic sensors 217A and 217B, the positioning sensors 218A and 218B, the RTCs 214A and 214B, the displays 219A and 219B, the microphones 221A and 221B, the speakers 222A and 222B, the processors 225A and 225B, the memories 228A and 228B, the telephone network communication devices 231A and 231B, and the network communication devices 220A and 220B are connected to each other via the buses 240A and 240B that connect the respective components.
  • the network communication devices 220A and 220B are connected to the antennas 223A and 223B that transmit and receive network communication signals.
  • position and orientation sensors 235A and 235B are the same as in the case of an HMD, and include PP cameras 236A and 236B, PP distance sensors 237A and 237B, acceleration sensors 215A and 215B, and gyro sensors 216A and 216B.
  • the PP cameras 236A and 236B may be the same as the outer cameras 211A and 211B.
  • the PP distance sensors 237A and 237B may be the same as the distance sensors 213A and 213B.
  • Memories 228A and 228B store programs 226A and 226B and data 227A and 227B.
  • Touch panels 230A and 230B are stacked on displays 219A and 219B as input interfaces.
  • FIG. 4A is a functional block diagram of the image capture terminal 1A portion of the video sharing system.
  • FIG. 4B is a functional block diagram of the display terminal 1B portion of the video sharing system.
  • the photographing terminal 1A includes a position and orientation measurement unit 301 that acquires sensor information from the position and orientation sensor and the distance measurement sensor and generates first correction information to be transmitted to the display terminal 1B, a photographing processing unit 302 that acquires and processes photographed video information from the camera, a transmission information generation unit 303 that transmits the first correction information and the photographed video information, and a communication control unit 304 that transmits and receives data with the display terminal 1B.
  • the first correction information will be described later.
  • the display terminal 1B includes a communication control unit 311 that transmits and receives data to and from the photographing terminal 1A, a position and orientation measurement unit 312 that acquires sensor information from a position and orientation sensor mounted on the display terminal 1B and generates second correction information that indicates changes in the position and orientation of the display terminal 1B, an image information storage unit 313 that stores the received photographed image information, a display image generation unit 314 that performs correction processing of the photographed image using the first correction information and the second correction information, and a display control unit 315 that controls display on the display.
  • the second correction information will be described later.
  • the video sharing system 100 is characterized by the display mode when the image information generated by the imaging terminal 1A imaging the object 7 is transmitted to the display terminal 1B for display.
  • the characteristics are described with reference to Figures 5A, 5B, 6A, and 6B.
  • each HMD may be replaced with a smartphone 5A, 5B.
  • a combination of an HMD and a smartphone may also be used.
  • a different type of mobile information terminal may be used as long as it has equivalent functions.
  • FIG. 5A is a diagram showing an example of the positional relationship between the shooting reference point 410, the shooting reference direction 420, and the shooting object 7 in the shooting space in which the shooting terminal 1A exists.
  • This positional relationship state will be called the shooting reference state 400.
  • the point where a straight line extending from the shooting reference point 410 in the shooting reference direction 420 hits the shooting object 7 will be called the object center point 470.
  • the photographing reference state 400 sets the photographing terminal position 411 when the user 2A is in a position where it is easy to work as the photographing reference point 410. Furthermore, the direction in which the photographing terminal 1A's front direction 430, which is the optical axis direction of the outer camera 111A, is mainly pointed during work is set as the photographing reference direction 420. Also, as the work progresses, the position where it is easy to work and the direction in which the photographing terminal 1A's front direction 430 is mainly pointed may change. In that case, the photographing reference state 400 is updated in response to an instruction from the user 2A.
  • FIG. 5B is a diagram showing a captured image in the shooting reference state 400 of FIG. 5A. It is a diagram showing a captured image of the shooting terminal 1A when the shooting terminal 1A and the shooting object 7 are in the positional relationship of FIG. 5A, the shooting terminal 1A is at the shooting reference point 410, and the front direction 430 of the shooting terminal 1A faces the shooting reference direction 420. In this positional relationship, the object center point 470 on the shooting object 7 is located at the shooting image center 450, which is the center of the shooting image 440.
  • the distance between the shooting reference point 410 and the object center point 470 in the shooting reference state 400 is defined as the shooting object reference distance 415.
  • the shooting object reference distance 415 is measured by the distance measurement sensor 113A or the position and orientation sensor 135A.
  • the shooting reference point 410 and the shooting reference direction 420 are measured by the position and orientation sensor 135A.
  • the shooting terminal 1A transmits information on the shooting reference point 410, the shooting reference direction 420, and the shooting object reference distance 415 to the display terminal 1B at least each time the shooting reference state 400 is set or updated.
  • FIG. 6A is a diagram showing an example of the positional relationship between the display reference point 510, the display reference direction 520, and the image of the object to be photographed 8 in the display space in which the display terminal 1B exists.
  • the state in this position is called the display reference state 500.
  • the positional relationship of the image of the object to be photographed 8 in the display space in which the display terminal 1B exists is as shown in FIG. 6A.
  • the position of the image of the object to be photographed 8 is the position where the user 2B feels that the image of the object to be photographed 8 is located in the display space, and is the display position as 3D in the 3D image.
  • the photographed image 440 is displayed on the display terminal 1B so that the position of the object to be photographed 7 based on the photographing reference point 410 and the photographing reference direction 420 is the same as the apparent position of the image of the object to be photographed 8 based on the display reference point 510 and the display reference direction 520.
  • display terminal 1B displays image 8 of the subject to be photographed so that user 2B sees it in the same position as if the subject 7 were viewed from the photographing reference point 410. If the occlusion relationship with other objects in the display space is also represented, the image will become more realistic.
  • the expression "image 8 of the subject to be photographed” refers to an image that appears to be located in the display space to user 2B, as described above.
  • the display terminal position 511 when the user 2B is in a position where the displayed image is easily viewable is set as the display reference point 510.
  • the direction in which the front direction 530 of the display terminal 1B is mainly directed while viewing the image is set as the display reference direction 520.
  • the elevation angle of the display reference direction 520 with respect to the horizontal plane is adjusted to the elevation angle of the shooting reference direction 420. This is to view the state of the shooting site with the same feeling as the user 2A.
  • the front direction 530 of the display terminal 1B refers to the direction extending vertically from the center of the display surface of the display 119B of the display terminal 1B or the surface on which the user 2B recognizes that a display is being performed. If the display area is not a shape whose center can be obviously defined, such as a rectangle, the center of the display surface is appropriately defined. In addition, there may be cases where you want to change the viewing position or the direction in which the front direction 530 of the display terminal 1B is mainly directed while sharing the image. In that case, the shooting reference state 400 is updated in response to instructions from user 2B.
  • FIG. 6B is a diagram showing a display image in the display reference state 500 of FIG. 6A.
  • the display image 540 of the display terminal 1B will be as shown in FIG. 6B.
  • the object center point 570 on the image 8 of the object to be photographed is located at the display image center 550, which is the center of the display image 540.
  • Fig. 7A is a diagram showing an example of the positional relationship when the position and attitude of the photographing terminal 1A have changed from the relationship between the position and attitude of the photographing terminal 1A shown in Fig. 5A.
  • the attitude of the terminal will be described using the photographing terminal 1A as an example.
  • the three-dimensional rotational position of a mobile information terminal in real space is called the attitude.
  • the reference for the rotational position is the photographing reference state 400.
  • the amount of three-dimensional rotation from the photographing reference state 400 when the photographing terminal 1A is in the general state shown in FIG. 7A is the amount of change in attitude of the photographing terminal 1A. This three-dimensional rotation amount is measured by the position and orientation sensor 135A.
  • this three-dimensional rotation amount is decomposed into the amount of change from the photographing reference direction 420 of the front direction 430 of the photographing terminal, and the photographing rotation angle 435, which is the rotation angle from the photographing reference state 400 around the optical axis of the outer camera 111A.
  • the optical axis direction of the outer camera 111A and the front direction 430 of the photographing terminal 1A are aligned in the same direction.
  • the amount of three-dimensional rotation from display reference state 500 when display terminal 1B is in the general state shown in FIG. 8A is the amount of change in attitude of display terminal 1B.
  • This amount of three-dimensional rotation is measured by position and attitude sensor 135B.
  • This amount of change in attitude is then broken down into the amount of change in front direction 530 of display terminal 1B from display reference direction 520, and display rotation angle 535, which is the angle of rotation from display reference state 500 around the vertical axis of the display surface of display 119B.
  • the direction of the vertical axis of the display surface of display 119B and the front direction 530 of display terminal 1B are aligned in the same direction.
  • the position 411 and the front direction 430 of the photographing terminal 1A shown in FIG. 7A are both shifted from the photographing reference point 410 and the photographing reference direction 420 set in the photographing reference state 400 shown in FIG. 5A.
  • the point where the line extending the front direction 430 of the photographing terminal 1A hits the photographing subject 7 is called the photographing front point 480.
  • FIG. 7B shows the captured image 441 when in the position shown in FIG. 7A.
  • the user 2A shifts his/her gaze and checks the meter 600 at the bottom left.
  • the capture front point 480 is located at the center of the captured image 441
  • the object center point 470 is located at the top right.
  • the position and orientation sensor 135A measures, as correction information, the change in position of the capture terminal position 411 from the capture reference point 410, the change in the front direction 430 of the capture terminal 1A from the capture reference direction 420, and the capture rotation angle 435, which is the rotation angle from the capture reference state around the optical axis of the outer camera 111A of the capture terminal 1A.
  • the change in terminal position from the shooting reference state 400, the change in the terminal front direction, and the rotation angle around the optical axis of the camera, measured by the shooting terminal 1A are pieces of correction information used to correct the display image 541.
  • the change in terminal position from the shooting reference state 400, the change in the terminal front direction, and the rotation angle around the optical axis of the camera, measured by the shooting terminal 1A are referred to as first correction information.
  • the change in the terminal front direction and the rotation angle around the optical axis of the camera are specific contents of the change in attitude.
  • the change in terminal position from the display reference state 500, the change in the terminal front direction, and the rotation angle around the vertical axis of the display surface, measured by the display terminal 1B, are referred to as second correction information.
  • the change in the terminal front direction and the rotation angle around the vertical axis of the display surface are specific contents of the change in attitude.
  • this first correction information is sent to the display terminal 1B together with the captured image information. Furthermore, the front direction 430 of the terminal includes vertical information. This also makes it possible to adjust the elevation and depression angles.
  • Fig. 7C shows an image on display terminal 1B when photographing terminal 1A is in the state shown in Fig. 7A and display terminal 1B is in Fig. 6A (display reference state 500).
  • object center point 570 is located at display image center 550 on display terminal 1B.
  • the photographed image information is corrected to offset the change in terminal position, the change in the front direction of the terminal, and the rotation angle around the optical axis of the camera, which are indicated in the first correction information, and is then displayed on display terminal 1B.
  • the image of the object to be photographed 8 transmitted from the photographing terminal 1A is displayed only in the photographed image range 490, and the area outside the photographed image range 490 is a missing area where no image data is displayed.
  • the missing area is, for example, a solid black screen area.
  • the missing area is illustrated with diagonal lines.
  • FIG. 7C a screen including a missing area is displayed on display 119B, but in FIG. 6B, the entire area of display 119B of display terminal 1B is the same as captured image range 490, so the two differ in that no missing area occurs on display 119B. Note that in FIG. 7C, as in FIG. 6B, object center point 570 is located at display image center 550.
  • FIG. 8A is a diagram showing an example of the positional relationship when the position and attitude of display terminal 1B change from the positional relationship (display reference state 500) of the position and attitude of display terminal 1B shown in FIG. 6A. Assume that the position and attitude of display terminal 1B are as shown in FIG. 8A.
  • FIG. 8B is a diagram showing the display image in the positional relationship of FIG. 8A.
  • display image center 550 is a point located at the center of display image 541 on the image 8 of the object to be photographed.
  • Display image center 550 is also a point where the image 8 of the object to be photographed comes into contact with the image 8 of the object to be photographed when display terminal 1B is extended in the forward direction, assuming that the image 8 of the object to be photographed is located at that position for user 2B in the space in which display terminal 1B exists.
  • the second correction information which is the change in display terminal position 511 of display terminal 1B from display reference point 510, the change in front direction 530 from display reference direction 520, and display rotation angle 535, which is the angle of rotation from the horizontal state around the vertical axis of the display surface, are measured by a position and orientation sensor 135B in display terminal 1B.
  • the display terminal 1B corrects the position and magnification of the captured image information based on the second correction information and the first correction information sent from the imaging terminal 1A, so that the display terminal 1B displays a display image 541 as if the imaged object 7 were being viewed from a position corresponding to the display terminal position 511 in the imaging space, as shown in FIG. 8B.
  • the shooting image range 490 may not be able to cover the entire display area of the display 119B of the display terminal 1B.
  • a configuration may be used in which a wide-angle camera is mounted on the shooting terminal 1A to narrow the missing area.
  • the image after the correction described above may be enlarged or reduced as appropriate on the display terminal 1B.
  • Fig. 9 is a diagram showing parameters used in the explanation of the image correction method.
  • Fig. 10 is a diagram showing the relative relationship of a captured image and a displayed image to a reference image.
  • Fig. 11 is an explanatory diagram of parameters used in a real space where a capture terminal exists.
  • Fig. 12 is an explanatory diagram of each parameter used in a real space where a display terminal exists.
  • video correction is performed in the following two stages.
  • First correction The position change of the photographing object on the image caused by the change in position and attitude of the photographing terminal 1A is returned to its original state.
  • Second correction A change in the position of the object to be photographed on the image caused by a change in the position and attitude of the display terminal 1B is reflected.
  • a reference image is considered to be intermediate between the image captured by the camera, i.e., the imaging terminal 1A, and the image displayed on the display 119B, i.e., the display terminal 1B.
  • the reference image is an image captured when an object in the outside world is captured by placing the imaging terminal 1A in the imaging reference state 400. In other words, it is the image in FIG. 5B.
  • each terminal is assumed to be in a horizontal state.
  • the reference state is a state in which the elevation angle remains the same and the housing is in a horizontal state.
  • the imaging range and image display range are rectangular, so the horizontal state is a state in which one side of the rectangle is horizontal.
  • a terminal coordinate system is set for each image with the center of the image as the origin, the image direction that is horizontal in the reference state as the X axis, and the direction perpendicular to the X axis as the Y axis ( Figure 9). If the definition is expanded to accommodate terminals whose image range is not rectangular, the image horizontal direction will be the direction on the image that corresponds to the horizontal direction perpendicular to the front direction of the terminal in the outside world when the terminal is held horizontal at the time of use.
  • the position of point P on the image is represented by coordinate values (X P , Y P ).
  • the angle at which the entire width of the image is seen from the terminal position of the photographing terminal 1A is the angle of view.
  • FIG. 9 shows the horizontal angle of view and the vertical angle of view. The angle of view may be different between the photographed image and the displayed image.
  • the distance between the photographed object 7 or the photographed object image 8 and the terminal position is L.
  • L is the distance between the photographed reference point 410 and the object center point 470, and is also the distance between the display reference point 510 and the object center point 570, which is the photographed object reference distance 415 (FIGS. 5A and 6A). This is represented as L 0 in FIG. 11.
  • L is the distance between the photographed terminal position 411 and the photographed front point 480 (FIG. 7A). This is represented as L C in FIG. 11.
  • L is the distance between the display terminal position 511 and the display front point 580 (FIG. 8A). This is represented as L D in Fig. 12.
  • L differs between images, the image range and apparent size of the subject 7 will differ depending on the ratio of L at the position of the subject 7 or the subject image 8.
  • the position of the subject image 8 refers to the position at which the subject 7 photographed by the photographing terminal 1A is displayed on the display terminal 1B as if it were located there.
  • the position of the object 7 to be photographed is the same for each image.
  • the photographed image will differ from the reference image depending on the position and attitude of the photographing terminal 1A.
  • the photographing reference state 400 the state in Figure 5A.
  • the position and orientation of the photographing terminal 1A deviates from the photographing reference state 400, the point that should be at the center in the reference image will deviate from the center in the photographed image.
  • the attitude of the photographing terminal 1A rotates around the optical axis of the camera and deviates from the direction of the photographing reference state 400, the photographed image will be tilted with respect to the reference image.
  • the displayed image will differ from the reference image depending on the position and orientation of display terminal 1B.
  • display reference state 500 the state in Figure 6A. If the position and orientation of display terminal 1B deviates from display reference state 500, the point that should be at the center in the reference image will deviate from the center in the displayed image. Furthermore, if the orientation of display terminal 1B rotates around the vertical axis of the display surface and the horizontal direction of the image deviates, the displayed image will be tilted with respect to the reference image.
  • a coordinate system for each image is defined.
  • the reference image it is the reference coordinate system ⁇ X (0) , Y (0) ⁇
  • the display coordinate system ⁇ X (D) , Y (D) ⁇ for the photographed image, it is the display coordinate system ⁇ X (D) , Y (D) ⁇ .
  • the superscripts (0), (C), and (D) represent the respective coordinate systems.
  • the photographing coordinate system and the display coordinate system are terminal coordinate systems fixed to the photographing terminal 1A and the display terminal 1B in their respective reference states, and change following changes in the position and attitude of the photographing terminal 1A and the display terminal 1B.
  • these superscripts will indicate which coordinate system the position expression is based on.
  • the relationship of the positional expression in each coordinate system is represented by the coordinate system origin position, the rotation angle from the horizontal direction of the coordinate system, and the distance L between the shooting object or the image of the shooting object and the terminal position.
  • the shooting coordinate system rotation angle is represented by ⁇ C .
  • ⁇ C refers to the shooting rotation angle 435 (FIG. 7A).
  • x (0) C is a vector notation of the position coordinate, and the same applies below.
  • the display coordinate system rotation angle is ⁇ D .
  • ⁇ D refers to the display rotation angle 535.
  • the rotation angle of the coordinate system is positive in the counterclockwise direction with respect to the reference coordinate system.
  • R( ⁇ ) is a matrix representing the rotation of the rotation angle ⁇ (counterclockwise is positive).
  • position expression will be in vector notation.
  • x (0) P (L D / L 0 ) R ( ⁇ D ) x (D) P +x (0) D ... (1)
  • (L D /L 0 ) represents the magnification of the apparent size due to the difference in the distance L.
  • x (C) P (L D /L C )R(- ⁇ C )x (D) P +(L 0 /L C )R(- ⁇ C )(x (0) D -x (0) C )...(3)
  • the corresponding point x (C) P may not be within the angle of view of the camera of the photographing terminal 1A.
  • the photographing terminal 1A acquires the photographing reference direction L0 , Lc , ⁇ c , and x (0) C, which serve as first correction information, and transmits the photographed image information together with them to the display terminal 1B.
  • the display terminal 1B acquires Ld , ⁇ d , and x (0) D, which serve as second correction information, and creates a display image by combining this with the information transmitted from the photographing terminal 1A.
  • L 0 is measured by the distance measurement sensor 113A as the distance between the photographing reference point 410 and the object center point 470 (FIG. 5A).
  • ⁇ C is measured by the position and orientation sensor 135A as the photographing rotation angle 435, which is the rotation angle of the photographing terminal 1A camera from the photographing reference state 400 around the lens optical axis.
  • the measurement of L C and x (0) C will be described below.
  • L C is obtained from the front direction 430 of the photographing terminal 1A. It may be measured as the distance between the photographing terminal position 411 and the photographing front point 480, but this may lead to unexpected errors, such as the photographing front point 480 hitting a protrusion of the photographing object 7. Therefore, the point where the plane including the reference object center point 470 and perpendicular to the photographing reference direction 420 intersects with the front direction 430 of the photographing terminal is set as the origin of the photographing coordinate system. The distance between the photographing coordinate system origin determined in this way and the photographing terminal position 411 is set as L C. x (0) C is also obtained as a coordinate value in the reference coordinate system of the origin of the photographing coordinate system.
  • the position and orientation sensor 135A measures the displacement r C of the photographing terminal position 411 at the time of photographing from the photographing reference point 410.
  • the rotation of the front direction 430 of the photographing terminal 1A from the photographing reference direction 420 is measured, and a unit vector u C of the front direction 430 of the photographing terminal is obtained.
  • the unit vector of the photographing reference direction 420 is u 0.
  • L 0 and u 0 are measured when the photographing terminal 1A is in the photographing reference state 400.
  • a three-dimensional coordinate system is considered in which the two-dimensional reference coordinate system is the XY plane.
  • the reference coordinate system plane is perpendicular to the photographing reference direction 420, and the X-axis direction of the reference coordinate system is set to the horizontal direction of the real space.
  • L C (L 0 -r C ⁇ u 0 )/(u C ⁇ u 0 )...(6)
  • equation (7) is obtained.
  • x (0) C r C + ⁇ (L 0 ⁇ r C ⁇ u 0 )/(u C ⁇ u 0 ) ⁇ u C ⁇ L 0 u 0 ...(7)
  • L C and x (0) C can be calculated from L 0 , u 0 , u C , and r C, so the first correction information that the photographing terminal 1A measures and transmits to the display terminal 1B can be L 0 , u 0 , u C , r C , and ⁇ C.
  • the second correction information L D and x (0) D obtained by the display terminal 1B will be described.
  • the same discussion can be made on the display terminal 1B side (FIGS. 8A and 12).
  • the direction vectors u 0 and L 0 from the display reference point 510 to the object center point 570 are those transmitted from the photographing terminal 1A.
  • the terminal position when the display terminal 1B is in the terminal position and terminal attitude that makes the display reference state 500 is set as the display reference point 510, and a coordinate system is set so that the vector from the display reference point 510 toward the object center point 570 is L 0 u 0.
  • the object center point 570 is the position at which the object center point 470 of the photographing object 7 is displayed as if it were located there.
  • first, position and orientation sensor 135B measures displacement rD of the display terminal position at the time of display from display reference point 510. At the same time, rotation of front direction 530 of display terminal 1B from display reference direction 520 is measured, and unit vector uD of front direction 530 of display terminal 1B is found. Then, using the same discussion as for imaging terminal 1A, the following equations (8) and (9) are found.
  • the first correction information measured by display terminal 1B may be u D , r D , and ⁇ D.
  • a timestamp may be added to the video information and the first correction information.
  • FIG. 13 is a flowchart showing the processing flow of the video sharing system according to the first embodiment.
  • the user 2A wears the photographing terminal 1A and starts it up. If the photographing reference state 400 is to be updated at the time of startup or during the video sharing state, the process branches to YES in the branch judgment of S01 and proceeds to processing of S02. Otherwise, the process branches to NO and proceeds to processing of S03.
  • the processor 125A of the photographing terminal 1A sets the photographing reference point 410 and the photographing reference direction 420 (u 0 ) in accordance with an input operation from the user 2A. After this setting, while keeping the photographing terminal 1A in the photographing reference state 400, the distance measuring sensor 113A measures the photographing object reference distance 415 (L 0 ) as first correction information, and the position and orientation sensor 135A measures the photographing reference direction 420 (u 0 ) (S02).
  • the position and orientation sensor 135A measures the displacement (r C ) of the photographing terminal position 411 from the photographing reference point 410, the front direction 430 (u C ) of the photographing terminal, and the photographing rotation angle 435 ( ⁇ C ) as correction information (S03).
  • the photographing terminal 1A photographs an image including the field of view of the user 2A using the outer camera 111A (S04).
  • the photographing terminal 1A transmits the image information and the first correction information to the display terminal 1B (S05).
  • the photographing terminal 1A returns to S01 and repeats the process until the termination condition is met (S06: No).
  • the termination condition is met (S06: Yes)
  • the photographing terminal 1A terminates the process.
  • the "termination condition” here may be, for example, an operation to shut out the photographing terminal 1A or an operation to terminate the video sharing app.
  • Display terminal 1B receives the image information and correction information from imaging terminal 1A (S11).
  • the branch decision in S12 branches to YES and proceeds to processing in S13. Otherwise, the branch decision in S12 branches to NO and proceeds to processing in S14.
  • the processor 125B of the display terminal 1B sets the display reference direction 520 according to the input operation from the user 2B (S13).
  • position and orientation sensor 135B measures displacement (r D ) of display terminal position 511 from display reference point 510, front direction 530 (u D ) of the display terminal, and display rotation angle 535 ( ⁇ D ) as second correction information (S14).
  • the processor 125B of the display terminal 1B corrects the received image information using the first correction information measured by the imaging terminal 1A and the terminal position and attitude information of the display terminal 1B measured in step S13, and displays it on the display 119B (S15).
  • the imaging terminal 1A determines an imaging reference point 410 and an imaging reference direction 420 based on the settings of an imaging reference state 400 in the imaging space in which the imaging terminal 1A is located. Then, the display terminal 1B determines a display reference point 510 and a display reference direction 520 based on the settings of a display reference state 500 in the display space in which the display terminal 1B is located. Then, the display terminal 1B displays the captured image 440 so that the position of the object 7 to be captured based on the imaging reference point 410 and the imaging reference direction 420 is the same as the apparent position of the image 8 of the object to be captured based on the display reference point 510 and the display reference direction 520.
  • the user 2B e.g., a work assistant
  • the display terminal 1B can view an image in a direction that the user 2B wants to see, close to the direction that the user 2A (e.g., a worker) is looking, by simply pointing the display terminal 1B in that direction, without being affected by the movement of the imaging terminal 1A.
  • the basic idea of setting the display reference state 500 is that when the user 2B is in a position and posture that is easy for him/her to work, the same image as the shooting reference state 400 is visible. If the orientation of the display terminal 1B is changed from that state, the user 2B can freely view the part he/she wants to see. However, in some cases, a request may arise to view the shooting target image 8 from a place far away from the same image as the shooting reference state 400 or from a different angle. In that case, the display terminal position 511 and the front direction 530 of the display terminal when the user 2B is in a position and posture that is easy for him/her to work may be shifted from the display reference state 500.
  • the relative relationship between the display terminal position 511, the front direction 530 of the display terminal, and the display reference point 510 and the display reference direction 520 of the display reference state 500 may be changed as appropriate or returned to the original state by an instruction from the user 2B.
  • the reference of the display rotation angle 535 may be shifted from the horizontal direction in the same manner.
  • the photographing reference state 400 is updated in response to an instruction from the user 2A.
  • the photographing reference point 410 and the photographing reference direction 420 may be changed continuously or in small increments.
  • the average position and attitude of the imaging terminal 1A may be set as the imaging reference state 400.
  • the average position of the imaging terminal position 411 may be set as the imaging reference point 410
  • the average direction of the front direction 530 of the imaging terminal may be set as the imaging reference direction 420.
  • part of the processing may be performed by a terminal paired with the photographing terminal 1A and the display terminal 1B, for example, a smartphone 5A, 5B, or a server.
  • the photographing terminal 1A is an photographing terminal capable of photographing the user 2A himself/herself, such as a smartphone
  • the photographing subject 7 may be the user 2A.
  • an image that is not affected by the movement of the terminal can be viewed.
  • the outer camera 111A in the above description should be read as the inner camera 212A.
  • One mobile information terminal may perform the processing of the photographing terminal 1A and the processing of the display terminal 1B, and each terminal may display the photographed image of the other mobile information terminal.
  • the two terminals can talk while looking at the surroundings that the other terminal is looking at, which deepens the level of communication.
  • it can be used like a TV phone in combination with the selfie mode.
  • This embodiment relates to an alternative image for an area that cannot be photographed by an imaging terminal.
  • the captured video is recorded on the display terminal 1B or a server (not shown), and if a missing area occurs in the displayed video, the video of the missing area is extracted from past video and the past video is inserted and displayed through synthesis.
  • This is useful when there is information that can be used as reference even in past video. For example, information that does not change, such as a description of the subject of the video, can be used as reference even in past video.
  • FIG. 14 shows an example of a display image in the second embodiment.
  • the processor 125B of the display terminal 1B extrapolates and displays a past captured image range 590 onto the captured image range 490.
  • the extrapolated image information may be image information of the entire captured space captured in advance by the capturing terminal 1A, or data of the captured space on a server (not shown) may be used.
  • FIG. 15 is a flowchart showing the flow of processing according to the second embodiment. Note that the same steps as those in the flowchart of FIG. 13 are given the same step numbers, and duplicate explanations will be omitted.
  • the processing of the photographing terminal 1A is the same as in the first embodiment.
  • display terminal 1B When display terminal 1B receives the video information and the first correction information (S11), it records the video (S21).
  • the shooting image range 490 included in the received video information is displayed as the display image 542.
  • new video information is received in step S12 from the second time onwards through the loop process, if the newly acquired shooting image range overlaps with the extrapolated previous shooting image range 590, the previous shooting image range 590 is replaced with the shooting image range 490 included in the new video information.
  • the previously captured image range is extrapolated and displayed, making it possible to display an image to be displayed to user 2B in a wider range than the newly received captured image range 490.
  • This makes it possible to link the capture terminal operated by the person capturing the work situation, etc., with the display terminal that receives and displays the captured image transmitted from the capture terminal, and to display an image wider than the captured image on the display terminal, without using a large fixed display device.
  • multiple captured images may be composited and displayed on the display device.
  • the photographing terminal 1A determines the photographing reference point 410 and the photographing reference direction 420 based on the setting of the photographing reference state 400 in the photographing space where the photographing terminal 1A is located. Then, the display terminal 1B determines the display reference point 510 and the display reference direction 520 in the display reference state 500 in the display space where the display terminal 1B is located. Then, the display terminal 1B displays the photographed image 440 so that the position of the photographing object 7 based on the photographing reference point 410 and the photographing reference direction 420 and the apparent position of the photographing object image 8 based on the display reference point 510 and the display reference direction 520 are the same position. Then, the recorded past image is also synthesized and displayed.
  • the user 2B of the display terminal 1B can view images in a wide range of directions that the user 2B wants to view in the vicinity of the direction in which the user 2A of the photographing terminal 1A is looking, without being affected by the movement of the photographing terminal 1A, by a natural operation of pointing the display terminal 1B in that direction.
  • each processing example may be independent programs, or multiple programs may constitute a single application program. Furthermore, the order in which each process is performed may be changed.
  • each function may be realized in part or in whole by a server.
  • the server may be any type of server as long as it can execute functions in cooperation with other components via communication, such as a local server, cloud server, edge server, or net service.
  • Information such as programs, tables, and files that realize each function may be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD, or may be stored in a device on a communication network.
  • a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD, or may be stored in a device on a communication network.
  • control lines and information lines shown in the diagram are those considered necessary for explanation, and do not necessarily represent all control lines and information lines on the product. In reality, it can be assumed that almost all components are interconnected.
  • a video sharing system configured by connecting a photographing terminal and a display terminal for communication
  • the photographing terminal includes: A camera and A first position and orientation sensor that detects a position and orientation of the photographing terminal; A first processor; a first communication device for transmitting and receiving data between the display terminal,
  • the display terminal includes: A display and a second position and orientation sensor for detecting a position and orientation of the display terminal; A second processor; and A second communication device that transmits and receives data to and from the photographing terminal,
  • the first processor of the photographing terminal In a photographing reference state set in a real space in which the photographing terminal exists, a position of the photographing terminal is set as a photographing reference point, and a direction in which the front of the photographing terminal faces from the photographing reference point is set as a photographing reference direction, and the photographing reference point and the photographing reference direction are transmitted to the display terminal; acquire a change in position of the imaging terminal with respect to the imaging reference point when the camera images the object to be photographed and an amount of change in
  • (Appendix 2) A video sharing method executed by communicating between a photographing terminal and a display terminal, In the photographing terminal, in a photographing reference state set in a real space in which the photographing terminal exists, a position of the photographing terminal is set as a photographing reference point, and a direction in which the front of the photographing terminal faces from the photographing reference point is set as a photographing reference direction; acquiring, in the photographing terminal, a change in position of the photographing terminal with respect to the photographing reference point when photographing an object to be photographed, and an amount of change in attitude of the photographing terminal with respect to the photographing reference state; a step of transmitting, in the photographing terminal, a change in position of the photographing terminal and an amount of change in attitude of the photographing terminal as first correction information to the display terminal together with photographed video information generated by photographing the object to be photographed; a step of setting, in the display terminal, a position of the display terminal as a display reference point in a display reference state set in a real space in which the display
  • Photographing terminal 1B Display terminal 2A: User 2B: User 3A: Access point 3B: Access point 4: Communication network 5A: Smartphone 5B: Smartphone 7: Photographing subject 8: Photographing subject image 100: Image sharing system 111A: Outer camera 111B: Outer camera 113A: Distance measurement sensor 113B: Distance measurement sensor 114A: RTC 114B: RTC 115A: acceleration sensor 115B: acceleration sensor 116A: gyro sensor 116B: gyro sensor 117A: geomagnetic sensor 117B: geomagnetic sensor 118A: positioning sensor 118B: positioning sensor 119A: display 119B: display 120A: network communication device 120B: network communication device 121A: microphone 121B: microphone 122A: speaker 122B: speaker 123A: antenna 123B: antenna 125A: processor 125B: processor 126A: program 126B: program 127A: data 127B: data 128A: memory 128B: memory 135A:

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Controls And Circuits For Display Device (AREA)
PCT/JP2023/012897 2023-03-29 2023-03-29 映像共有システム及び映像共有方法 Ceased WO2024201814A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202380096256.4A CN121002887A (zh) 2023-03-29 2023-03-29 影像共享系统以及影像共享方法
PCT/JP2023/012897 WO2024201814A1 (ja) 2023-03-29 2023-03-29 映像共有システム及び映像共有方法
JP2025509427A JPWO2024201814A1 (https=) 2023-03-29 2023-03-29

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/012897 WO2024201814A1 (ja) 2023-03-29 2023-03-29 映像共有システム及び映像共有方法

Publications (1)

Publication Number Publication Date
WO2024201814A1 true WO2024201814A1 (ja) 2024-10-03

Family

ID=92903650

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/012897 Ceased WO2024201814A1 (ja) 2023-03-29 2023-03-29 映像共有システム及び映像共有方法

Country Status (3)

Country Link
JP (1) JPWO2024201814A1 (https=)
CN (1) CN121002887A (https=)
WO (1) WO2024201814A1 (https=)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013206322A (ja) * 2012-03-29 2013-10-07 Sony Corp 情報処理装置、情報処理システム及び情報処理方法
WO2015122108A1 (ja) * 2014-02-17 2015-08-20 ソニー株式会社 情報処理装置、情報処理方法及びプログラム
US20180181196A1 (en) * 2016-12-22 2018-06-28 Samsung Electronics Co., Ltd. Method for displaying image, storage medium, and electronic device
JP2018182570A (ja) * 2017-04-14 2018-11-15 コニカミノルタ株式会社 視線情報共有方法および視線情報共有システム
WO2021053805A1 (ja) * 2019-09-20 2021-03-25 マクセル株式会社 情報端末装置および位置認識共有方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013206322A (ja) * 2012-03-29 2013-10-07 Sony Corp 情報処理装置、情報処理システム及び情報処理方法
WO2015122108A1 (ja) * 2014-02-17 2015-08-20 ソニー株式会社 情報処理装置、情報処理方法及びプログラム
US20180181196A1 (en) * 2016-12-22 2018-06-28 Samsung Electronics Co., Ltd. Method for displaying image, storage medium, and electronic device
JP2018182570A (ja) * 2017-04-14 2018-11-15 コニカミノルタ株式会社 視線情報共有方法および視線情報共有システム
WO2021053805A1 (ja) * 2019-09-20 2021-03-25 マクセル株式会社 情報端末装置および位置認識共有方法

Also Published As

Publication number Publication date
JPWO2024201814A1 (https=) 2024-10-03
CN121002887A (zh) 2025-11-21

Similar Documents

Publication Publication Date Title
US10841570B2 (en) Calibration device and method of operating the same
US11328493B2 (en) Augmented reality screen system and augmented reality screen display method
JP2018201123A (ja) 画像処理装置、画像処理方法、プログラム
CN114127794A (zh) 信息终端装置以及位置辨识共享方法
WO2022180459A1 (en) Image processing method, recording medium, image processing apparatus, and image processing system
US12586457B2 (en) Home device position marking method and electronic device
CN113384880A (zh) 虚拟场景显示方法、装置、计算机设备及存储介质
US12075168B2 (en) Portable device
JP2017212510A (ja) 画像管理装置、プログラム、画像管理システム及び情報端末
JP2019185757A (ja) 画像処理装置、撮影システム、画像処理方法、及びプログラム
US11928775B2 (en) Apparatus, system, method, and non-transitory medium which map two images onto a three-dimensional object to generate a virtual image
WO2024201814A1 (ja) 映像共有システム及び映像共有方法
US20240340403A1 (en) Head mount display, information processing apparatus, and information processing method
KR20180119281A (ko) 이동 단말기 및 그 제어 방법
JP2024037496A (ja) 画像処理装置、プログラム及び画像処理方法
JP6816614B2 (ja) 画像出力プログラム、画像出力方法および画像出力装置
KR101729923B1 (ko) 스크린 영상과 증강현실 영상을 이용한 입체영상 구현 방법 및 이를 실행하는 서버 및 시스템
CN114371819B (zh) 扩增实境屏幕系统及扩增实境屏幕显示方法
JP2017126868A (ja) 画像表示システム、情報処理装置、画像表示方法及び画像表示プログラム
JP2013206365A (ja) 画像処理装置、画像処理方法及びプログラム
CN110443841B (zh) 地面深度的测量方法、装置及系统
WO2024224608A1 (ja) 映像共有システム及び映像共有方法
CN117135456B (zh) 图像防抖方法与电子设备
CN114201028A (zh) 扩增实境系统与其锚定显示虚拟对象的方法
JP2020170400A (ja) 情報処理装置、情報処理方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23930452

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025509427

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025509427

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: CN2023800962564

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23930452

Country of ref document: EP

Kind code of ref document: A1