Groove Jones was brought on board to assist the Toyota Lexus team in developing an advanced AR system, known as the Lexus Augmented Reality Kit (LARK). This interactive application demonstrates how Augmented Reality can aid in the education and training of Toyota team members to understand the features of a vehicle. The App is the future of digital user manuals for cars. the LARK system was part of the Innovaiton section of the Toyota Experience Center (TEC), showcasing the connection between digital and physical product interaction, bridged through the use of spatial awareness and contextual data.
By using the Lexus Augmented Reality Kit (LARK), the experience moves beyond traditional 2D screens and turns the physical vehicle—or even a printed representation of it—into an interactive interface. This “phygital” approach allows a user to touch a physical button in the car while simultaneously seeing a digital overlay that explains its internal mechanics or purpose.
This synergy is further enhanced by two distinct modes of interaction that redefine how a customer learns about a product:
-
Object Recognition (The Physical Anchor): In the first mode, the AR system uses machine learning to identify specific physical components of the dashboard in real-time. This transforms the physical car into a “live” user manual. Instead of looking away to a book or a separate screen, the digital information is anchored directly to the physical part the user is touching, creating a seamless cognitive link between the tactile feel of the car and the digital knowledge about it.
-
Virtual Synthesis (The Digital Extension): In the second mode, the experience can detach from the physical object entirely. By placing a computer-generated, 3D version of the dashboard into a user’s immediate environment, the TEC allows for “interaction without presence.” This demonstrates a future where the digital product experience is no longer tethered to a showroom floor, allowing users to explore high-fidelity physical details through a purely digital medium from anywhere in the world.
Video of Launch Event
The Lexus AR app uses machine learning to “learn” what each of the features and controls are so that it can “identify” what is displaying on the screen. We also use Augmented Reality to place tap-able icons at each button so that the user can bring up the features and sample videos of each button.
Application Installation at the Toyota Experience Center
Multinational automotive manufacturer Toyota just opened its first visitor attraction in North America at their headquarters in Plano Texas. The 44,000sq ft (13,400sq m) Toyota Experience Center (TEC) is Toyota’s first and only comprehensive museum space in North America. The TEC exists to educate its visitors about Toyota’s past, present and future as a mobility company.
The museum features a variety of interactive exhibits and displays, all designed to reflect on the people behind the Toyota brand.
The LARK system recognizes features inside the vehicle, and digitally displays tags that, when tapped, activate information about that product feature. A user can then tap on to view related media which includes product videos and visual demonstrations. Think of it as a digital User Manual that recognizes the vehicle and provides product information at your fingertips.
The LARK system being demonstrated at the TEC works in two modes. Mode one is an Image Recognition system that triggers from a 2D print of a Lexus dashboard, displaying AR tags over selected features of the car. Mode two allows the user to place a computer-generated, 3D version of the car’s dashboard and center console wherever they choose. A person can then learn about the vehicle features without having the actual vehicle in front of them.