Search Issue Tracker

Duplicate

Fixed in 2018.4.X, 2019.4.X

Votes

14

Found in

2019.2.17f1

Issue ID

1207427

Regression

No

PhotoCaptureFrame Fails to Provide Camera Matrices on HoloLens 2

XR SDK

-

1. With the same Unity App built on top of Unity PhotoCapture and PhotoCaptureFrame API, it is able to take photo and retrieve CameraToWorld and CameraProjection matrices on HoloLens 1. The app works fine taking photo data, however, always fails to provide CameraToWorld and CameraProjection matrices on HoloLens 2.

2. How we can reproduce it using the example you attached
1) Have Windows SDK 18362 and VS2019 installed
2) Load UnityPhotoCapture scene into Unity 2019.2.17f1
3) Choose Universal Windows Platform as platform, and ensure capabilities WebCam, Microphone, Location and SpatialPerception are enabled, and Virtual Reality Supprt - Windows Mixed Reality is enabled as well.
4) Build and deploy the app onto Hololens 2
5) Once app gets launched, air-tap to take a photo and watch debug log. It prints out "Failed to get camera to world matrix!" and "Failed to get camera projection matrix!"

  1. This is a duplicate of issue #1297267

    PhotoCaptureFrame provides wrong camera matrices on HoloLens 2 (claimed fixed, not working

  1. Resolution Note (2020.2.X):

    Duplicate of 1297267. The newer bug has a repro project that shows more edge cases of the issue.

Log in to vote on this issue

Comments (32)

  1. MarionV

    Oct 22, 2020 08:29

    Hi. Unity did a release (2019.4.13) and it says : XR: Fixed rotated projection when using PhotoCaptureFrame on HoloLens 2. I hoped it corrected this massive bug, so I did a quick test and CamToWorld stays the same between different tests on a single session. Is it normal ? Has someone else tested it ?

  2. fyyang

    Oct 16, 2020 00:01

    I got same matrix from TryGetCameraToWorldMatrix all the time with 2019.4.6f1 on HL2? Any updates?

  3. McFrolic

    Aug 20, 2020 14:30

    I can confirm this issue and it's a huge blocker for me. Reproduced in Unity 2018.4.26f1. PhotoCaptureFrame.TryGetCameraToWorldMatrix and PhotoCaptureFrame.TryGetProjectionMatrix returning true now, but matrices are not correct. When calculating rays from 2D PhotoCaptureFrame into the 3D world by using these matrices, on HoloLens 1 the rays are correct, while on HoloLens 2 the rays are totally wrong.

    When taking two PhotoCaptureFrames with different viewing angles, it can be seen that the HL1 CameraToWorldMatrix has different values while the HL2 CameraToWorldMatrix is nearly the same. That can't be correct.

    Example:

    PhotoCaptureFrame's CameraToWorld matrices from both HoloLenses viewing north:

    HL1:

    0.96491 -0.06117 -0.25534 -0.08812
    0.01769 0.98542 -0.16920 0.15655
    -0.26197 -0.15875 -0.95193 -0.25989
    0.00000 0.00000 0.00000 1.00000

    HL2:

    0.01313 -0.99939 -0.03227 0.00257
    -0.99966 -0.01239 -0.02304 -0.05001
    0.02263 0.03257 -0.99921 -0.01091
    0.00000 0.00000 0.00000 1.00000

    PhotoCaptureFrame's CameraToWorld matrices from both HoloLenses viewing south:

    HL1:

    -0.96757 0.05874 0.24569 -0.24733
    0.03725 0.99513 -0.09121 0.11127
    0.24985 0.07910 0.96505 -0.14452
    0.00000 0.00000 0.00000 1.00000

    HL2:

    0.01313 -0.99939 -0.03227 0.00257
    -0.99966 -0.01239 -0.02304 -0.05001
    0.02263 0.03257 -0.99921 -0.01091
    0.00000 0.00000 0.00000 1.00000

  4. MarionV

    Aug 19, 2020 07:36

    vfatuanpv : The link is :

    https://issuetracker.unity3d.com/issues/photocaptureframe-provides-wrong-camera-matrices-on-hololens-2.

    Unity support told me that the link is not yet active but will be shortly.

  5. vfatuanpv

    Aug 19, 2020 04:39

    MARIONV: Excuse me, Could you give me a link you reported, I want to follow with you, It's so important with me

  6. MarionV

    Aug 07, 2020 09:10

    I've sent a bug report for this problem, I hope they will fix it.

  7. vfatuanpv

    Aug 07, 2020 08:06

    Confirmed: TryGetCameraToWorldMatrix returns true but Value correctly on HoloLens 1 and wrong on Hololens 2.

  8. vyke234

    Aug 07, 2020 01:42

    same as WALKRAMIS. I keep getting the same matrix when I use TryGetCameraToWorldMatrix. The values I get using TryGetProjectionMatrix are also messed up even though the function returns true this time.

  9. francescogdev

    Aug 06, 2020 09:29

    Confirmed: values are messed up on HoloLens 2 but correct on HoloLens 1

  10. pocce90

    Aug 06, 2020 08:14

    Same as WALKRAMIS, but the same code works well in hololens 1.

Add comment

Log in to post comment

All about bugs

View bugs we have successfully reproduced, and vote for the bugs you want to see fixed most urgently.

Latest issues

See all new issues