Apple critics have been quiet lately.
The company is worth more than $900 billion after beating Wall Street’s expectations in its November 2 earnings report. The iPhone 8 is selling better than expected. Consumers are lining up to buy its most expensive iPhone ever, the X. And the iPad just might be making a comeback.
Tim Cook is talking like a visionary, positioning himself and Apple on the cusp of changes in technology and human experience. For instance, Cook recently declared on an Apple earnings call that augmented reality is “mainstream” and that “Apple is the only company” that could have made augmented reality mainstream.
His comments evoke Mark Zuckerberg’s bold announcement that Facebook intends to get one billion people to use virtual reality. And, like Zuckerberg, Cook is being ambitious, considering that only 12 percent of the U.S. population is expected to use AR at least once a month in 2017. But there is reason for AR backers to be optimistic: usage of AR is growing by 30 percent over 2016 according to eMarketer.
Apple’s strategy to accelerate the uptake of augmented reality is to provide a development platform for the creation of AR content and to rely on popular Apple devices as Trojan Horses to deliver that content to consumers.
But to realize the potential of augmented reality for widespread consumer and corporate use, Apple might need to do more — such as the creation of an augmented reality headset.
Augmented Reality Breaking Through
Augmented reality refers to an experience that alters our perception of reality by overlaying computer-generated content on to a physical space. Augmented reality is being used in businesses ranging from hospitals to amusement parks to train and entertain by enhancing our worlds with digital content such as holograms and 3D objects with which we can interact. In the automotive industry, augmented reality might enhance driving by overlaying content such as signage on a driver’s windshield, reducing the need for the driver to strain to read street signs while navigating. For AR to break through to more mainstream consumer use, the experience needs:
- Great content.
- A ubiquitous, user-friendly delivery mechanism.
Apple provides the latter through the manufacture of its devices and is enabling content creation by providing the necessary tools and media platform.
The Apple fan boys want Apple to deliver something magical and different with AR – like the launch of a pair of AR smart glasses. But instead of delivering a big bang, Apple is creating smaller, less breathtaking moments that may have long-lasting impact. For example:
- In June, Apple launched ARKit, Apple’s AR development platform for iOS mobile devices. It’s the toolkit that developers need to create AR apps that will work on your iPhone. ARKit is Apple’s answer to Google’s Tango, an AR platform for Android devices.
- iOS 11, released September 19, gave Apple mobile devices (including iPhones going back to the 6s model) access to apps created with ARKit. With ARKit, iOS 11 became “the world’s largest augmented reality platform,” according to the Apple website.
The launch of app developer toolkits matter – very much. Apple’s decision in 2007 to open up the iPhone to software developers (through a software development kit) turned the iPhone into a delivery device for the critical native apps whose existence iPhone owners now take for granted. Had Apple closed the iPhone to third-party app development, the device would have remained an isolated island with far less functionality and popularity than it has today.
An App Explosion
Within weeks of the launch of iOS 11, more than 1,000 apps with AR feature have hit the App Store, according to Cook. For example:
- Strava, the social fitness-tracking app, released Fitness AR, which includes the ability for runners or bicyclists to visualize their routes on a 3D terrain map.
- IKEA released Place, which makes it possible for shoppers to see how IKEA furniture might look in their living spaces. With the app, users overlay 3D models of furniture into their physical spaces to test for fit, which takes some of the risk out of buying a sofa or bookshelf before carting it home.
- On November 2, Amazon, not to be outdone by IKEA, released a similar app for shoppers to overlay on to physical spaces renderings of products available from Amazon, which mitigates against one of Amazon’s few vulnerabilities: product returns.
- AR MeasureKit turns your iPhone into a measuring tool for flat surfaces, angles, cubes, and room dimensions without requiring you to crawl around or stretch to reach inconvenient spots. The device also acts as a leveler.
AR MeasureKit and IKEA Place are among the most popular free ARKit-only apps worldwide, according to SensorTower. Utility apps are the second most popular category, with games being the most popular app on the Apple Store. The most popular free ARKit-only app is AR Dragon, and the highest grossing is a game known as The Machines. Apps that provide diversionary entertainment such as My Very Hungry Caterpillar are also popular.
Early reactions to these apps underscore the emergent nature of AR on the iPhone. Forbes contributing writer Charlie Fink characterizes many of the early AR apps as “not ready for prime time” and, at best, “delightful demoware.” And for all the media fanfare that IKEA Place has achieved, the 120 customer ratings/reviews to date suggest plenty of room for improvement.
In addition, AR apps require a lot of battery power on any smartphone (including the iPhone and its competing brands). The battery drain may act as a winnowing factor, forcing users to pick and choose only the most compelling AR apps given their demands on phone power.
Great consumer-facing AR content has a long, long way to go. And the battery drain is a definite threat.
Changing the Workforce
My Very Hungry Caterpillar notwithstanding, Tim Cook believes AR has more near-term potential than virtual reality to change how people live. During the November 2 earnings call, he said, “Simply, we believe AR is going to change the way we use technology forever.”
Indeed, AR was already changing the way people use technology in the workplace before ARKit came along. For instance:
- Physicians are using AR for more precise, complicated forms of surgery by overlaying computer-generated enhanced images that assist the surgical team.
- NASA uses AR to help astronauts explore the terrain of Mars by projecting the Martian surface on to physical training rooms. (The simulation is helping NASA research the possibility of astronauts traveling to Mars by the 2030s.)
- Retailer Sephora is using AR to turn the purchase of cosmetics into more playful yet functional experience in which shoppers can virtually see how cosmetics look on their faces before they make a purchase.
Augmented reality, like virtual reality, will probably realize its promise – and popular uptake — first with businesses using the experience to improve functions such as shopping, training, and surgery. A simple Google search uncovers many more examples than the few I’ve cited. For instance, Michael E. Porter’s and James E. Heppelmann’s Harvard Business Review article, “Why Every Organization Needs an Augmented Reality Strategy,” provides a thoughtful discussion of AR’s uptake in the workplace.
In addition, Porter and Heppelmann articulate why augmented reality holds such promise to change our everyday lives:
More broadly, AR enables a new information-delivery paradigm, which we believe will have a profound impact on how data is structured, managed, and delivered on the internet. Though the web transformed how information is collected, transmitted, and accessed, its model for data storage and delivery—pages on flat screens—has major limits: It requires people to mentally translate 2-D information for use in a 3-D world. That isn’t always easy, as anyone who has used a manual to fix an office copier knows. By superimposing digital information directly on real objects or environments, AR allows people to process the physical and digital simultaneously, eliminating the need to mentally bridge the two. That improves our ability to rapidly and accurately absorb information, make decisions, and execute required tasks quickly and efficiently.
Apple would do well to partner with businesses to develop more corporate AR experiences built off ARKit, as Facebook is doing for virtual reality with the launch of Oculus for Business, a bundled set of VR products designed by Oculus (which is owned by Facebook) to help businesses build virtual reality applications.
A key to building a beachhead for ARKit will be getting the software embedded in corporations like Walmart and McDonald’s for uses such as training and sales support. As is the case with VR, large companies are fertile ground for making AR more popular through their employees. (McDonald’s and Walmart employ 3.7 million between them.) The business absorbs the cost of the equipment. The employees learn new skills and become more accustomed to using augmented reality outside the workplace.
But software may not be enough. The more sophisticated forms of AR needed for training and complex functions such as surgery require advanced headsets such as Microsoft’s HoloLens. Apple does not offer such a device. As such, Apple’s penetration of the business world is limited to companies willing to design applications using iOS devices supported by ARKit. Developing a more powerful device for AR — as Apple is rumored to be doing — might be in Apple’s future.
Even still, forming relationships with key corporate partners is something Apple does well. In other areas, Apple has already demonstrated an ability to forge partnerships that accelerate the uptake of its products and software. For instance, Apple has been steadily embedding the Apple Watch into the healthcare industry by making available the Apple ResearchKit and CareKit application development software frameworks. ResearchKit and CareKit provide a software infrastructure to development apps for wellness care, diagnostic care, and medical research.
Apple’s One-Two Punch
AR lacks the compete immersion that virtual reality provides. But many forms of AR do not require the clunky, expensive headsets that VR does — although, as noted, advanced AR experiences certainly do. But as the wildly successful AR game Pokémon GO and the popularity of Snapchat filters have demonstrated, anyone with a mobile phone and the right software can use augmented reality.
Although Apple will likely need to create a device if the company wishes to penetrate the corporate world for advanced training, everyday usage of augmented reality will come from people adopting AR on the devices they own already as opposed to buying headsets. Apple has the right idea by focusing on iOS devices for consumer applications.
Augmented reality has a long way to go before becoming mainstream, and Apple’s leadership of the market is not assured. As noted, the battery drain caused by AR apps is an impediment (for any device manufacturer, not just Apple). Apple’s one-two punch of development tools (through ARKit) and a user-friendly, ubiquitous delivery device (iOS embedded on iPhones) gives the company an early advantage. But competitors such as Google are not going away.
Now bring on the content.