AR glasses from Apple – the future of augmented reality in smartphones?

Based on materials Phone Arena

Apple Glass vision by Martin Hajek of iDrop News.

Last year, Facebook-owned Oculus introduced the Oculus Quest 2, an Android-based VR system powered by the Qualcomm Snapdragon chipset. That is, in fact, it is a self-contained VR system based on technologies created for smartphones. There is no reason to connect your Oculus Quest 2 to your computer, as all Android apps and games that support augmented reality are already available. It even has its own virtual browser. And more recently, downloads from third-party sources and the ability to work with unoptimized Android applications are supported, which at the same time, with a certain probability, even work normally in virtual reality mode. With some reservations, you get a full-fledged virtual reality experience, but more importantly, after revision, the system turns into a full-fledged version for augmented reality in Android OS.

Well, what does Apple and the future have to do with it if everything relates to the present and to the Android OS? Well, the answer is simple – the system in Oculus Quest is still raw, and Apple is famous for taking someone’s raw but promising ideas and creating a ready-made and user-friendly product based on them. Why not assume that this is exactly what should happen with augmented reality systems? Indeed, in the past, as we know, iPod, iPhone, iPad, Apple Watch and AirPod were not the first devices of their type on the market, but they were exactly the ones that set the development trend. If Apple does this with an augmented reality device, history will repeat itself.

Hundreds of millions of users will one day learn about the very existence of augmented reality, and then they will start using it in everyday life. Other smartphone manufacturers will not be able to fail to follow this trend and will start offering customers their versions of augmented reality glasses.

But back to today. Augmented reality is still at the beginning of its journey, but look at what you can try right now and appreciate the potential that can really change the way you look at things. Remember how the Internet and smartphones made such a revolution in consciousness. And a new one is just around the corner.

Custom AR systems aren’t perfect yet, but very promising

Oculus Quest 2 can “see” your hands, recognize gestures and thus turn them into controllers. But only in some scenarios

The Oculus Quest 2 is an Android based gaming VR system based on smartphone hardware and is therefore relatively affordable. Since it was created as a virtual reality game system, then, being in virtual space, you do not see the objects around you – instead, only the digital environment.

But some time ago, Quest 2 received the Passthrough feature, which recently left beta testing and is now available to all users. This feature allows you to use two of the helmet’s four cameras to show your real environment as 3D models, almost as if you were looking at it with your own eyes.

The function is far from ideal – there is graininess, the background is generally monochrome, and sometimes with interference. But that’s okay. After all, the Oculus Quest 2 cameras weren’t designed to show your surroundings, but rather to track your movements and controller positions. However, the Oculus developers added augmented reality functionality and did it in a decent way. Enough to see two things – how AR works on Android and what the future holds for smartphone technology.

For now, the Android AR experience that you get from the Oculus Quest 2 shows you a real-world environment with three floating windows on top of it. For example, you can open two browser windows, one for YouTube videos, one for Wikipedia, and the third for a file browser on your device to view your saved photos, videos, and music. Yes, all this cannot be said to work without flaws, but look at how it already works and assess the prospects. The illustration below demonstrates how this looks. Note that the background is a simulation, as there is no way to take a screenshot of what the cameras are seeing in the Oculus Quest 2.

Android AR multitasking for Quest 2.

The augmented reality system in Android is still at an early stage of development, so there are enough bugs, but, at the same time, there is more than enough interesting interaction experience. Now imagine that after a while, a much more polished Apple AR system appears.

And keep in mind that Apple is not developing a helmet, but augmented reality glasses, so you will not suffer from the quality of the transmission of the environment by imperfect cameras, but will look at it with your own eyes through the lenses of the glasses, onto which transparent digital content will be projected. A more fantastic option looks like the version that Apple will immediately offer a device that projects an image directly onto the retina of the eye. In any case, you will not get a device-modeled environment, which depends on the capabilities of the cameras and the processor, but you will see everything with your own vision. As if looking at the world through ordinary glasses. The difference is that AR glasses can project digital content on top of what you see, augmenting reality. Hence, in fact, the name.

What will Apple’s AR glasses look like? Will they need an iPhone to run?

Vision Apple Glass by Martin Hajek.

Ultimately, it is assumed that Apple is developing a device that will be exactly like regular glasses, they can be folded if necessary, and all the necessary components will be in the frame. So this is a standalone device that will supplant the iPhone someday.

But at the moment this is difficult from an engineering point of view, so most likely the first augmented reality device from Apple, which is rumored to be shown in 2022, will be completely dependent on the iPhone. Taking full advantage of the iPhone’s capabilities, Apple’s augmented reality glasses will delegate all necessary calculations to the smartphone, while they will only have to scan the environment and display the results of the calculations. Presumably, they will have a LiDAR and a camera.

For Apple, this will be a brilliant move that will sell both another device for the iPhone and the smartphones themselves. Now we know that there is little practical use from lidars in the iPhone and iPad, but the company is not abandoning this technology, which means it is accumulating experience for something more.

So what can you do with augmented reality glasses?

Spotify AR app for Magic Leap corporate augmented reality device.

Imagine chatting with friends without a smartphone – you see messages from them in the corner of your field of vision, while drinking your morning coffee, and typing reply messages with your voice.

Imagine watching a podcast doing your morning exercises, but the image does not rush due to head movements, but constantly in front of your eyes, while the sound comes from the speakers mounted in the frame.

Or imagine traveling abroad where road signs and signs or any other writing right in front of your eyes are translated into your language as soon as you look at them.

Or you navigate using Apple Maps, but instead of constantly peering at the screen of a physical smartphone, you see directions in the form of arrows on the real road, and street names float in the air.

This is by no means fiction, your iPhone can already do all this, so you already have the magic of augmented reality. V iOS 15you can already point the camera at the text and get an offer from your smartphone to translate what you have written. If you open Apple Maps somewhere (although the function does not work everywhere), you can get immersive directions, scan buildings and translate road signs …

AR navigation in iPhone. Now imagine her, but without the iPhone

Apple’s augmented reality glasses will simply save you the hassle of holding a physical device and looking at its screen instead of directly interacting with the environment. Your iPhone will remain in your pocket, but it will be busy calculating everything that the augmented reality glasses see and hear. And all the information you need will appear right before your eyes.

AR magic provides an unusual experience that will most likely change our lives one day. And this day is not so far away. The first mainstream devices will probably have the most basic functionality, but over time, Apple AR Glasses (or their potential competitor) will become a necessity for orientation in the real world, for social interactions, entertainment and more. Information from the Internet and your smartphone will mix with the real world, but will no longer be tied to the device in your pocket.

Some businesses are already using augmented reality devices.

And since AR technologies in the future, albeit a very distant one, have the potential to completely replace our smartphones, tablets and smartwatches, we can also consider the future of mobile technologies in them. Already, they are attracting the attention of enthusiasts, making it possible to make some familiar things more convenient and easy to use. And if augmented reality devices take the next step, they could displace all familiar digital devices from everyday life.

Why exactly should Apple take the coup?

In short, Apple CEO Tim Cook has spoken repeatedly over the years that Apple is experimenting with augmented reality. And the company has already succeeded in promoting devices that have not previously “taken off” in others.

Apple has at its disposal billions of dollars, which it spends, including on research in virtual and augmented reality technologies for more than one year, as we can judge from published patents and the fact that new ones continue to appear. But patents cannot appear indefinitely; real devices must follow. The first patents related to augmented reality devices from Apple appeared in 2017, so you can take it as a starting point for starting development. But it would be more correct to evaluate some of the words of Tim Cook, said a year or two before.

Here are a selection of quotes from Tim Cook starting in 2016, that is, over the past five years, as well as statements about the promise of VR.

  • We have invested and continue to invest heavily in this area. We are betting heavily on augmented reality in the long run, we believe there is a lot to offer to our customers, and the business has great business opportunities.
  • There is virtual reality and there is augmented reality – both are incredibly interesting. But my personal point is that augmented reality is the bigger of the two, at least for now.
  • I believe that a significant part of the population of developed countries, and ultimately all countries in general, will come to get an augmented reality experience every day. It’s like eating three meals a day; it will become part of your nature. Many of us live in our smartphones, the iPhone is, I hope, very important to everyone, so AR will become significant too.
  • Augmented reality will have to wait because there are some significant technological challenges. But it will come, it will spread widely, and when it does, we will wonder how we did without it. As we are now wondering how we used to live without smartphones.

So augmented reality is the future, and Tim Cook’s words confirm that this is exactly what Apple believes, sees a commercial perspective and is working to make this future a reality. A reality in which our world around us will be supplemented with digital information right before our eyes. And, most likely, Apple products will start the triumphant march of AR devices. And it will happen very soon.

P.S. This view is the personal opinion of one of the Phone Arena editors Rado Minkov. We do not know how to look into the future, but it seems that very soon we will find out how correct this forecast was.

Source: — Все о мобильной технике и технологиях by

*The article has been translated based on the content of — Все о мобильной технике и технологиях by If there is any problem regarding the content, copyright, please leave a report below the article. We will try to process as quickly as possible to protect the rights of the author. Thank you very much!

*We just want readers to access information more quickly and easily with other multilingual content, instead of information only available in a certain language.

*We always respect the copyright of the content of the author and always include the original link of the source article.If the author disagrees, just leave the report below the article, the article will be edited or deleted at the request of the author. Thanks very much! Best regards!