)
)
What inspired me to develop an AR app?
I have always had a massive fascination and love of mountains; the higher the better, and there’s nothing better for me than getting as much information as possible about the peaks around me.
A year and a half ago, I moved up to the West Coast of Wales in order to live with my girlfriend, and since then have worked remotely for CUBE. From our town, we have a view over cardigan bay to Snowdonia, and I am fascinated by the fact we can see so many mountains; often peeking over the horizon as if islands far out at sea…
But, not knowing what I’m actually looking at has been very frustrating, even with the best compass in the world I could only take a rough guess at what the mountains were. That’s why I spent my week of Innovation Time developing an augmented reality app with ARKit and CoreLocation.
)
Just one of the spectacular views that inspired my project and sparked my interest in building an AR App.
Introducing ARKit and CoreLocation:
Last year Apple released ARKit, which is a framework to allow mobile app development companies to more easily create Augmented Reality experiences in their apps! My idea was simple, and inspired by a library an ex-cube (Andrew) has worked on since ARKit was released: ARKit + CoreLocation.
CoreLocation is a piece of software that lets developers determine the location of different users, using wifi and GPS to provide a (somewhat inaccurate) geolocation for each mobile device running the app. This use of ARKit and CoreLocation together meant that I could use super accurate data based on the local-world of AR, with the scalability and wider-ranging data from CoreLocation.
Although It’s Android counterpart ARCore is another great piece of software for developing AR apps, I wanted to focus on building the app for iOS first, mainly because I’m an iOS developer! But also because I only had a week.
Check out the video below introducing ARKit and explaining augmented reality for iOS.
What does the Peak-Finder AR app do?
The app recognises mountains in the distance, providing information on the mountain name and height of the peak, as well as the location (even if it’s not completely in-sight) of the mountain either in the distance or over the horizon.
The app also provides a feature that lets you project these mountains onto nearly any surface that you’re looking at, using mobile cameras and augmented reality to create a miniature 3D model of each mountain peak.
Bringing these aspects together means that the Peak-Finder app can be used to identify different mountains in the distance, provide information on these like the distance and height of the peak, give you your exact position and distance from the mountain, and take a closer, more detailed look through the 3D model.
The challenges of building an AR app.
What I wanted to build was an app which positioned map pins on a real-world camera view of the user’s surroundings at the summit of the highest mountains around, to do this I faced a few challenges:
1) Locating the user in the real world.
This is important because if we are going to place pins on mountain summits, we need to know where the user is in the real world. We do this using a very similar technique to what Andrew uses in his library.
First of all, we get the user’s initial location using Apple’s CoreLocation APIs (these use the devices GPS to pinpoint where they are in the real world). Then as the user moves through the ARKit world, we re-calculate their latitude and longitude (geographical real-world coordinates) by offsetting that initial location by the distance they have moved in the ARKit world…
Which results in a much more accurate position than the GPS chip provides.
)
2) Placing the pins...
Once we have the user’s location in the real world, we can start with the mean task of placing pins in the ARKit world at the tops of the mountains, well… almost.
Because the ARKit world is managed by Apple and can be quite resource-intensive (burning through battery life and CPU/GPU power). We place the pins instead on a sphere around the user at a particular distance so the pins appear as if they are at the top of the mountain but are in fact floating mid-air on a visual line between the user and the peak of the mountain.
This kills two birds with one stone: the pins appear to the user as if they are placed at the peak, and we avoid an issue caused by the fact that ARKit automatically makes things smaller the further they are from the device, so we don’t have to worry about scaling up our pins to always appear the same size.
)
3) Modelling mountain peaks in augmented reality:
Once we had the pins placed, the next challenge was modelling the selected peak in 3D so the user can view and explore it in detail from their current location. My first port-of-call here was trying to find elevation data models for the whole world. I found a very good free elevation data model from the Japanese Space Agency.
Unfortunately, even with the data in hand, this was my first experience generating 3D models and so I needed to do some learning! Or did I. Just by pure coincidence, at this point I discovered a fantastic SDK provided by MapBox which does a lot of the hard-work of generating the 3D surface for you.
With a small bit of work we then had mountain peaks rendering in AR for the user to explore!
)
The future of our AR Peak Finder app?
Whilst this approach does work well when the user has good GPS signal and is stationary, there are a few improvements and pieces of functionality that could be made to make this even more killer.
The altitude information from the GPS chip can be very inaccurate, which can mean that whilst the pins appear in the correct direction, they don’t appear at the correct height. Luckily google provide a nice API which returns a higher-accuracy altitude for any given coordinate which would make this a lot better.
At the moment all mountains are simply hardcoded, but there are sources out there which provide elevation and positional data for all mountain ranges to bring this to the whole world.
If you have any ideas, advice or inspiration around the Peak-Finder app, let me know @3SIDEDCUBE!
Published on 13 April 2018, last updated on 15 March 2023