CVPixelBuffer data layout in iOS 12.1 ARKit

bug notes: ...frame.capturedImage in iOS 12.1 started to show green lines and wrong video frame.
Fix was to take real CVPixelBufferGetBytesPerRowOfPlane instead ofCVPixelBufferGetWidth
and stretch image with UV coords  based on the difference between the two.
https://lists.freedesktop.org/archives/gstreamer-devel/2012-November/037921.html








Both the stride (bytes of padding added to each row), as well as the
extended rows (rows of padding at the bottom or top of the buffer) are
important.
The extended rows are what changed in iOS6; you can find out how many rows
of padding are added to the buffer using:
CVPixelBufferGetExtendedPixels(pixelBuffer, &columnsLeft, &columnsRight,
&rowsTop, &rowsBottom)
In the example you gave (Medium preset on iOS6) you should be seeingrowsBottom = 8.

The stride is effectively CVPixelBufferGetBytesPerRowOfPlane() and includes
padding (if any).
When no padding is present CVPixelBufferGetBytesPerRowOfPlane() will be
equal to CVPixelBufferGetWidth(), otherwise it'll be greater.

 

Advertisements

Towards spatial interactive

at myVR I have been working last months  to bring into the life ARKit and ARCore prototypes/apps using mMap SDK.  At HxGN Live 2018 they have been presented as part of the Hexagon’s Xalt.  You can see the demos  on the page here: http://www.myvr-software.com/xrpocs/

Looking back, one of the best decision for me was to go native in 2011 for iOS app “Spatial Reader”  to make mobile apps. It was not because iOS was more difficult and challenging, rather potential of the device as whole  that can be exploited only by going deep into the platform. This tight integration Apple is pursuing  is about interactivity, user experience and simplicity, and  in this ARKit case – literally ‘Spatial Interactivity’.

For HxGN live we have also  used anamorphic image to represent real excavation – the type of image and projection that make perfect 3D illusion from certain point of view.

MapKit with ARKit and overlays

Flyover mode in Apple Maps allows  AR/VR style interaction.  This is not by default available for iOS developers using underlaying MapKit/ARKit technology.  However it is possible to test it and the following short video is about this proof of concept – viewing cadastral maps (iKatastr)  in VR like experience on iPad .  Btw. Flyover mode on iOS 11  has some strange handling of overlays – described here so loading of tiles is little bit tricky. The iOS 10 version was much more better (check the video here)