Letting Workers Qualify Themselves in AR, with AR Expert’s Dr. Björn Schwerdtfeger

XR for Business

Getting future workers excited for the jobs they might have tomorrow can be challenging, especially when many young workers tend to enjoy challenging themselves with new tasks. Dr. Björn Schwerdtfeger says that AR training can allow those workers to qualify themselves for all sorts of tasks, and have fun doing it, to boot.

[Transcript coming shortly]

Alan: Hey, everybody, and thanks for joining in on the XR for Business Podcast with your host, Alan Smithson. I’m really excited today. I have Dr. Björn Schwerdtfeger from Germany. He has more than 15 years experience in augmented reality. Together with the German industry, he’s evaluated almost every idea for AR in applications in the industry. He’s been a co-inventor of Pick-By-Vision at TU Munich. And during that time — when computers for AR glasses were still carried in large backpacks — Björn holds a PhD in industrial augmented reality as a serial entrepreneur. And among other things, his company, AR Experts, is advising about a third of Germany’s most important production companies, and is shaping their augmented reality roadmaps. You can learn more about them at ar-experts.de. And they have another product that they’re gonna be talking about today. It’s ar-giri.com. Björn, welcome to the show, my friend.

Björn: Yeah. Welcome, Alan. Nice to meet you. Nice to meet you online. I’m looking forward to this podcast.

Alan: It’s so exciting. The work that you’ve been doing over the last few years — like a decade and a half — is really starting to come to fruition now. I mean, all of the hard work that you and your team have done to evangelize a technology that — let’s be honest — 15 years ago, the technology really wasn’t ready for the market. Tell us, how did you get into this, into AR?

Björn: It was actually quite funny, while still studying at the university, computer science, and then somewhere else, augmented reality which popped up. And someone had a demonstrator, where someone took some glasses and glued a webcam — we had external webcams earlier — just hot-glued to some glasses and using some [unclear] stuff and highlighting it. I think it was just a cube. A virtual cube… And it was so fascinating that you can bring this computer interface into the real world. Quite a long time ago. But it was really nice.

Alan: Björn, did you say there was a webcam hot-glued to a pair of glasses?

Björn: Exactly. That’s how we did augmented reality 15, 20 years ago.

Alan: Amazing. You are one of the OG, the originals of this industry. You’ve been building and advising brands and companies around their strategy for production. What is the one thing in augmented reality right now that you’ve seen the most ROI?

Björn: It’s probably… we’ve seen a lot of companies trying to do everything. Basically every single one of us have tried to out, in the last three decades, and failed with it. And we’re figuring out what is actually the core of augmented reality. And the core of augmented reality is not– it’s not a measurement tool, it’s not a tool for everything. It looks like a display, and it is a good display. But where its core is, where it’s so good in, is in communication. It displays communication and augmented reality is big. It’s so much more close to your reality, that perception is getting much better. So what you tried to communicate with exosheets, nice PowerPoints; it’s getting so closer to the user with augmented reality. And they figured out that the communication got so much better using augmented reality — using *good* implemented augemnted reality, it’s quite important — you can do a lot of mistakes there. But this is helping so much. And that’s why you’re seeing currently augmented reality mainly in marketing, because marketing is a form of communication. You see it in museums, because museums are also a form of communication. And then work training, because training is also a lot of communication. Communicating the knowledge someone has to another user is key. It’s the same in the museum. A few people have some knowledge of what has happened the past, of the stories. This is a tool to improve the way you’re telling stories, basically. That make sense to you?

Alan: Makes total sense. So, when you say communications, are you meaning, I put on a pair of glasses and an avatar pops up and starts talking to me, or maybe a video screen? Run a certain example of what that would look like in an industrial space.

Björn: It’s– augmented reality, it doesn’t do a lot, but what it does, it really improves the way we communicate things. An avatar can pop up, but it’s also communicating technically-complex things, like someone has constructed a car, or a lot of people have constructed a car, and a lot of other people need to produce that car. There’s a lot of technical challenge you need to talk about. You can make those things visible. AR is a good tool to make things visible. Also, for training processes, AR is really good thing, a tool to make things visible. And also museums, you’ve got some boring things there, but then you could put the story over it as a new layer, and the story is being told through augmented reality. And that’s why it’s so nice.

Alan: I recently read a book called “The Age of Smart Information” by Mike Pell, and he talks about how information now is just information. I mean, we have access to the world’s information — I can Google pretty much any details — but it just gives me the information. It’s not in context to my world. It’s not in context to me. I can ask Google questions now; Voice, Amazon, Alexa, and Google Voice. But really, what they’re– what you’re discussing here, what you’re talking about is being able to look at a machine and the machine providing you with the story on how to fix it, or how to deal with it. Is that what I’m gathering here?

Björn: Well, for part of it, and also bringing this information, registering to the real world. That’s what you’re saying. Yeah, you’ve got the information in Google, but Google has information somewhere in servers, you can pick it up using your smartphone. But actually, you want to look at a machine or an exhibition, and you want to get information which is there exactly for that machine. And that’s where augmented reality can help, and where it makes so much more fun; finding the information so much faster and with making it fun, basically.

Alan: I recently read an article that came out. It’s a study by IBM saying that over 120 million people are going to need reskilling and upskilling as AI and robotics replace human workers. 120 million people. In my opinion, AR and VR are the fastest ways to do this. But you mentioned something that really struck me. You mentioned that it’s a lot more fun. And I think, if you look at education as a system, it’s competing with Hollywood movies and AAA video games. If we don’t start making education faster and more fun, then I think we’re gonna start falling behind, as we need to reskill people as fast as humanly possible. What are your thoughts on this?

Björn: We’re talking about a lot of topics now. So, one part is that 120 million people need to be reskilled. Seems that, I guess it’s one billion people working in production. One billion people producing things every day. I see that are a lot of more people need to be reskilled every day, because also the jobs are getting more complex. So when you’re working with clients, you always try to do an internship first with the customer. So you’re trying to do the job, which we try them. They get the support. We then analyze the situations and realize, hey, this guys doing a job since 10 years, 20 years, but now life is getting more complex, because they need to handle the more complex machines. The tasks are getting more complex, and they need to do so much more work, so much more different work. The world’s moving faster and the environments that you’re working in is getting so much more complex. The answer is always “Okay how can you fix it in the future?” The always say that you need more people. It’s always an answer, because those people are not thinking about making it easier to work on the more complex tasks. At least in Germany, we face a problem. You cannot plant more people for doing your job.

Björn: This is universal, my friend. They just did a study recently — I’ll have to put it in the show notes — but they did a study of 3,000 youth in the US and China. And they asked them what jobs they wanted. They gave them seven jobs to choose from. And in China, the number one job was astronaut. In America, the number one job was YouTube influencer. Think about that for a second: in Western society, our kids would rather try to be a YouTube influencer traveling around the world on Snapchat and Instagram, than actually contributing to a business. And the mindset of that, I think is going to start haunting us in the future, where AI is being taught to grade five children, and grade school in China.

Björn: And also, China used to be the workbench of the world 20 years ago; everybody was proud and rising up to get a good job. And now, as you’re saying, everybody wants to be an astronaut. What we figured out is, there’s some old people who like their job, but they don’t find new people for the actual boring jobs. What we figured out is, we did a lot of– obviously a lot of interviews with the workers, and they’re saying, when you’re doing this with glasses and I can self-qualify myself for that workplace? That’s really interesting, because on the one hand, you don’t put production workers on some different things because then they’re liable to do a lot of mistakes. But on the other hand, they want to do so many different jobs. But it won’t be so expensive to qualify them for so many different jobs. But for the business on the one hand, they need to do it. And on the other hand, they want to do it. So when you give them the possibility to qualify themselves for so many more, different jobs, that’s making it interesting.

Alan: If you look at people’s LinkedIn — especially younger people — they’re changing jobs every three years. And that from a process standpoint — from a factory, if you’re running a factory, you need somebody to work on a machine, and every three years you’re having to change it out — and there are people that are working there now, but the average age of people working in industry is escalating above 50. And people are starting to retire en masse. So you have this huge group of people retiring from jobs that they’ve been in for 20, 30, 40 years. People coming into those jobs, they’re looking at them going, “Well, I only want to do something for three years, most. And I want to try something new. I always want to be challenged.” And I think you mentioned something earlier — and I thought it was bang-on — is that we can make training fun. And by doing that, you’re not only being able to train people on new jobs fast, so they can feel like they’re always growing. But I think also you said that these jobs are becoming more and more complex, which I think is actually a benefit to everybody, because if it’s more complex, it’s actually more challenging for the learner or for the person working. And I think that’s really what gets people excited, is a real challenge in life. What are your thoughts on that?

Björn: As you’re saying, it’s a better fit for everybody. You can look at some jobs and say, okay, I need more people for this. Perhaps that’s not a good fit for everybody, because more people means more stress, and more cost. And everybody’s just doing one job, and people want to do different things. Or you look at the other side — you give people the possibility that they are getting the power to do more complex jobs, and different jobs — qualify themselves. Be more proud of what they can do, being more interested in that they can do different jobs. From the business side, that’s more flexible workers, they make less mistakes. They, say, remember 70 percent more. So it’s a better fit for everybody. If you look at this from a broader perspective, the benefit is — as with most things — is survival of the fittest. That’s the English saying for it, right? “Survival of the fittest,” Right?

Alan: But I think now we have the ability to– imagine, 20 years ago, if you wanted the facts about something, you had to look them up. There was no– well, I guess the Internet’s been around, but it was really difficult to do it. Now, I don’t even have to pull my phone out; I can just ask Google and every answer is there. So, it’s not really about what you know anymore. It used to be what you know, and what you knew gave you an advantage. But it’s no longer about that. It’s about how you apply that knowledge. And I think this is where augmented reality becomes a tool, like nothing we’ve ever had. You did a PhD in industrial augmented reality. What did that entail? Cause I know that that was a big part of your life. What does a PhD in industrial augmented reality mean, or what does it look like?

Björn: You need to look quite holistically at augmented reality. You cannot simply say, “oh, this improves a job,” but you also need to find a way how it improves the job. Industry is using things only if they work. So, does it work from the technical side — does it really do the job? If they work from the business side, does it save you money? And if it works for the people? If you don’t find a solution for all three areas — technology, people and business — it’s not working.

Alan: That is pretty much– if nobody takes anything else away from this podcast, that is it. You got to have the technology that works. It has to serve the people. And it has to make good business sense.

Björn: Exactly. And then there are also a lot of other obstacles. What was funny in the last years was — particularly in the last year — quite often sitting in management rounds, and the managers always were saying, “oh, we cannot give it to the workers. They are not accepting it and they don’t like it. We’ve got old workers. They cannot work with it. They were rejecting a lot of technology.” But if you work close together with people, with the workers, and figure out what they really need — what really helps those workers standing there, also the old guys — it’s really age-independent things. “It’s cool. When can we get it?”

Alan: It’s so true. You mentioned that– I had somebody else in the podcast, and they were saying, “Look, you really need to engage. You need three people in a conversation about rolling this out. You need the very high levels — you need maybe the CEO or somebody in the C level — to champion to say, yes, we’re gonna do this, you have my full support, go for it. Then you need an internal champion from the management level, who’s going to keep the communication between the C levels to keep the funding going for the project. And then you need somebody who’s actually going to be using it. Somebody who’s on the factory floor really doing this. And by having the buy-in of the three different levels, that’s how you get real change.” Is that what you’re seeing?

Björn: Yeah. Also, I think the most important — you need you need he OK from CEOs, from the innovation manager, but what you really need is commitment from the users who will really tell you what they need. It needs to be a solution for the workers. If it doesn’t work with workers, they are simply destroying your glasses and they’re not going to use them anymore.

Alan: Yeah, I can see that. So if you look at augmented reality in the enterprise, are you seeing more of this moving to glasses, or is it tablet-based for now? What’s being used most, and what is kind of delivering the most effective ROI?

Björn: You see some stuff happening on iPads. And it’s always quite nice. It’s scalable. Everybody says it’s a more robust technology. But if you’re doing a deep dive into technology — into the perception of information — it’s very different with glasses. You have a window to the world; you’re basically standing in your house. You’re looking through the window into the nice world. It’s like there’s some distance. What’s cool about augmented reality is that there’s this real-world overlay and it’s the distance between the point where you need information. It gets information presented. It’s going to zero. And this effect, you only have those glasses. So that’s why we’re seeing more and more people working on glasses. Also doing a lot of rollouts with glasses, for sure. Now you see a lot of rollouts for smart glasses, which is already — we don’t call it augmented reality in Germany. I know Americans do — but it’s good that it’s happening. So you see a lot of rollouts with smart glasses. We hope to see also more rollouts with the augmented reality glasses, and a lot of VR glasses getting rolled out — AR glass are not currently not getting rolled out because no one’s currently shipping AR glasses. There’s Magic Leap–

Alan: Still a challenge.

Björn: Yeah, but there are a lot of things are solved. The Hololens line. This device is like 4 years old. So yeah.

Alan: Yeah.

Björn: Everybody’s got the HoloLens 2, but they’re also waiting for some competitors [to start] shipping their glasses. Because from the general point of view, the technical challenge which is AR, there are some recent companies that have solved those problems. But it’s a matter of time. It’s a matter of months now, that Microsoft is shipping a new HoloLens; some other companies are shipping glasses. NReal is going to be shipping some glasses. Magic Leap is putting out their next generation of glasses and also shipping to the European market for sure — we have the glasses here, but we cannot currently roll out those glasses, because they don’t even have certifications for the European markets. So these are the next-level things which need to happen now; the glasses getting all the certification for the market.

Alan: It’s interesting, Björn, you mentioned– the first thing you mentioned was technology, then people, then has to make business sense. I think we’re still struggling with the technology side. We have glasses. We’ve proven the business cases; they work. And it’s just a matter of “Oh, you’re shipping them at scale now.” I think over the next six months, we’re going to see an absolute tsunami of technology being introduced to the marketplace. You mentioned three different types of glasses. You’ve got overlay glasses — or ones that kind of show you a heads-up display — like RealWear, for example, who just raised $80 million. And those are not really augmented reality glasses, they’re more heads-up display, similar to Google Glass. And then you’ve got VR headsets. And then you’ve got AR glasses. And the latter — the AR glasses — are the ones that are able to look at a machine and project images on top of it in context to that, in the real world. And VR has its place, heads-up displays have their place. But really, the magic is in the these AR glasses. Is that what you’re saying?

Björn: Exactly. And at the end, has the biggest benefit also, because the way you’re perceiving information, it’s made better if you get it presented in a stereoscopic way. That’s what glasses are doing. They benefit this way. But if you want to achieve this 70 percent of memory effect — that works really good, that people really are remembering so much more information because this is information in place — you need to do it with glasses.

Alan: I don’t know if you can give any specifics around companies that are deploying this. Are there any companies that you know that have deployed AR glasses at scale? Like, real AR glasses? Or are we still in the pilot purgatory of this?

Björn: Not really in scale yet. So, scale is like thousands of units. There are some companies using several thousands of devices, but really count as a problem. No devices are available, and the last Hololens was shipped one year ago. Now it’s like, all the glasses are breaking, because they so old. So you cannot roll out 1,000 glasses, because you cannot ensure that they will hold for such a long time until you get a replacement. That’s currently not happening. But I think it’s a matter of time. And also, the companies, the glass makers are fulfilling all the IT requirements that you can integrate into those glasses, and large infrastructures that can have property-wide management certifications, you’ve got health certifications, and all that stuff. But we think, from the technology point of view, it’s actually solved… well, not every glassmaker has solved every piece of puzzle.

Alan: Yeah, I think that’s interesting. You look at all the different glass manufacturers and they all have different parts solved.

Björn: Technology’s not a problem. There are many possible companies because we are scouting for all the AR/VR glasses. So in the last three years we’ve been reviewing, I think, 228 glasses. 228.

Alan: [chuckles] Holy crap.

Björn: That’s really quite a lot. For most glasses, half of them are shipped. For most of the other glasses, we’ve figured out, what is missing there? They all have different focus; what they want to be. You’ve also seen — which is quite sad, actually — we’ve seen the Meta glass, for example, quite early. And it was looked over by physicists and optical experts who said, “there’s fundamental problems here. Why the hell are they doing it?” And one year later or two years later, they went bankrupt. It’s the same device. But it’s so complex, augmented reality — also, virtual reality — and the companies also need to learn this. But we’ve got the feeling there are a few companies now who’ve learnt a lot, and will be able to ship a lot of glasses by the beginning of next year or so, I guess.

Alan: All right. So let’s talk about specifics. What are, in your opinion, the top five AR glasses? Out of– you reviewed 228 pairs of them, which is incredible. I would love to get that information to share with the listeners, if possible. But yeah. What are your top five, then?

Björn: I mean, the bigger ones you probably know as well. Yes, sure. Microsoft tries to be the leader of market, but they’re not shipping. It doesn’t show up in the top five company because they don’t ship. Magic Leap for sure. I mean, everybody’s saying Magic Leap is not so good, and they oversold what they ship. But a few things about Magic Leap are still better than Hololens 2.

Alan: Like what?

Björn: Wearer comfort is still better with Magic Leap than HoloLens 2. Because in the end, it’s way more lightweight.

Alan: Yeah, because they took the compute power off of the headset. Have you tried the Hololens 2 on?

Björn: Yeah, for sure. It’s really good. From the control, it’s way better than the HoloLens. The interaction is really good — they placed the HoloLens 1 on so many heads, and they always needed to explain a lot of things.

Alan: Yeah, yeah. That little pinch to click, nobody gets that. Anybody over 30 doesn’t get that right. We’re kind of in this weird place, where it’s like we know where the ROI is, companies want to deploy it, and we can’t [laughs] For whatever reason.

Björn: And then there are problems with a second row of companies won’t ship until the beginning of next year. I kind of was like, you know, all those companies. NReal doing quite a good job.

Alan: Yeah. NReal really impressed me as well. NReal is a spin off from Magic Leap. They took all of the best parts of the Hololens and Magic Leap idea. They’re like, “hey, really, what we need is just a lightweight display with a bit of tracking, and that’s it. We don’t need 8 cameras, and we don’t need a computer on the head, and all this.” And so they run it through USB-C — which now actually this week you can plug it in to your computer, I think they just announced — and you can also plug it into your phone, so your phone becomes the compute device, which makes the glasses much, much cheaper. I think they’re shipping at $599. Or that’s what they’re taking pre-orders for. I can tell you right now, we’ve done a lot of VR and AR development, and we used to haul around a big huge computer to do demos for people. We’d bring around, set up the computer, wire it up, put up the sensors, all of this. And I got the Oculus Go, thinking, “Oh, this is gonna be great for VR.” And it was just underwhelming. It was only 3 degrees of freedom. You could look around. But it overheated. The battery didn’t last very long. It was glitchy. And so when the Oculus Quest came out, we kind of said, “We’ll just wait, it’s not going to be as good.” But I recently tried it about a month ago, and I was very, very impressed with the Oculus Quest. As a VR headset that is standalone, the tracking is amazing. And now they just introduced a plug-in, where you can actually, via Wi-Fi, stream your computer to the Oculus Quest, so you can have computer-based rendering and graphics pushed to the Quest, which now makes the Quest an incredible tool for VR. I mean, it’s not AR, but man, the technologies behind these things really are getting much, much more impressive over the next little bit. So we’ve talked about Hololens, Magic Leap, NReal. What are some other ones that are kind of popping up in the top of your head that maybe are shipping in the near future?

Björn: We’ve gotten a lot over the years, so I’ve got a huge heap of glasses next to me.

Alan: Got a big pile?

Björn: Yeah.

Alan: You’re going to start a museum, Björn.

Björn: We called it– it’s a kind of museum. Yesterday, by the way, I was at the Deutches Museum — Germany’s technology museum — and they had the Oculus DK1. It was looking so good, because it’s looking so old already — it’s only 6 years old. But it was already looking so old. This technology has improved so much.

Alan: Yeah. The DK1 was the first VR headset I ever put on my head, and I remember looking at it. Somebody put it on, they had these big headphones. I thought, “Man, I don’t know what this thing is, but it looks ridiculous. [laughs] So big!”

Björn: [chuckles] Yeah, it’s like scuba diving.

Alan: Yeah, it was ridiculous. And I got to try, was it the Pimax? The one that’s 8K and it looks like a big V. Oh my goodness. It felt like I strapped a fishbowl on my head.

Björn: Yeah, but now it’s getting smaller, actually.

Alan: Yeah.

Björn: It’s small. And then also some other glasses are getting smaller. So technology’s advancing quite a lot.

Alan: You know what’s interesting about mixed reality? The AR glasses are going from a glasses standpoint, where you can see the real world and then you’re kind of putting holograms on top. But another company — Varjo from Finland — they’re taking a different approach. They built a really, really good VR headset and then they use pass-through cameras to create the mixed reality, or AR effect. And I tried it and I thought I was pretty skeptical, because usually when you have pass-through cameras, it’s nauseating. It makes you very sick — or makes me very sick. And so I put it on. And we had the pass-through cameras. I looked at my hands. And first thing you do is look at your hands, to see if it’s real time. And it was really, really accurate. It didn’t make me sick. It looked proper. And then being able to bridge that gap between, OK, you’re in VR — or sorry, in AR — you can see the whole world and then all the sudden they’re adding virtual layers to it. And I went from being in the real world with a car in front of me, to being in a completely virtual space. It was incredible. It’s maybe not practical for certain applications, but for applications where you need to see the real world and you can wear a big, bulky headset connected to a computer, I think it’s great.

Björn: It has some benefits. Varjo’s doing a great job, also has visors with a rich display, which helps a lot. Varjo is doing a great job, particularly with their retina center display. It’s amazing, what they are doing. It already helps a lot because you can see much more clearer situations in the industry to evaluate things. That’s quite good. What I didn’t get was a story with the see-through glasses. I saw it. I tried it. They’re doing a good job but still see-through. We had also the glasses of a Canadian startup. I forgot to name that it’s the same. They were acquired by Apple.

Alan: Oh, Vrvana!

Björn: Vrvana, yeah.

Alan: Yeah, out of Montreal.

Björn: Yeah. Yeah. We’re having basically the second device that they ever shipped.

Alan: Oh, very cool. You know why Apple bought them?

Björn: My guess was, as they were quite good in the latency and [unclear]. What was it?

Alan: There was those two things. But the one thing that they did — that nobody else could figure out — was occlusion from a single camera source. And if you look at the new ARKit system, the new ARKit allows you to do occlusion. And for people that don’t know what that means, if I’m looking in AR glasses and I look at a hologram that’s 3 feet away from me and somebody walks in front of that, it should know that they’re walking in front of it and to block it out as they walk through. If they walk behind it, it should walk behind it. But in most AR, it doesn’t recognize the depth. So as somebody walks through it, the object just becomes really big because it’s now projected on top of them instead of into the real world. And these guys solved that. And if you look at the new ARKit release that just came out, that’s actually one of the things that’s embedded in it now. So I think that’s one of the reasons they did that, because occlusion from a single camera is very, very difficult.

Björn: I saw the demo of the ARKit was showing this occlusion. I was really impressed. I was really impressed that this was working because I know about all the technical challenges behind it. And I was questioning whether it would only work with a stereoscopic iPhone, or a structured light sensor. But it’s not. It was only done by a single camera. This was looking so good, they a person walking around, you could see this person was walking in front of or behind the table. And this is amazing for augmented reality, because we test the technology to make the augmented reality look really good, to be more immersed to it.

Alan: Yeah, there’s nothing worse than somebody walking through your hologram when you’re trying to do something. [laughs] It’s very distracting.

Björn: Going back to the Vrvana device and media see-through. That’s the scientific term we’re having for it, media see-through augmented reality, in comparison to optical see-through. Media see-through part has a lot of other obstacles. So you’re having the frame rate issues, the latency issue. So, saying “is it real-time” when you move your hands, which is very important.

Alan: That’s what I think Varjo got really well. They nailed it. I mean, it was imperceivable, the latency.

Björn: That’s quite important. Then you having the amount of between black and white.

Alan: The contrast, yeah.

Björn: Contrast. That’s important. I mean, if you’re in a dark room, that’s fine.

Alan: How many people work in a dark room, though? Not very many. I think HoloLens 1 was pretty bright, and had some good colors and had a good contrast. But I think we need to do better, because most people work in very bright environments.

Björn: Yeah, but it’s even more complex when you’re looking at the biggest issue, because the cameras have low contrast and it’s only just to a certain area of environment. So what you can see is the monitor of the device, it’s only part of the world. If you’re standing in front of the window, you can’t help that it’s getting quite bright, or it’s getting dark and you can’t see what is outside the window. That’s the challenge.

Alan: I think it’s gonna be a challenge. For now, we need to be in windowless rooms for that.

[laughs]

Björn, I want to switch gears, because we have a little bit of time left and I want to switch gears and talk about AR Giri.

Björn: That’s our approach for training. For worker training, for training processes. So we figure it out in 2016 — a long time ago — we were asked to make some training apps. We did a lot of innovations at the assembly line, along with interviews with the workers, creating a lot of prototypes. And then yeah, we went live with some system and performance tests and we get random people coming in and we taught them, using Hololens, how to assemble a car engine and everybody managed it. In general, everybody managed it. And afterwards a guy came out of the one day experiments and say, “Hey, I never, ever assembled a car engine — I just assembled a car!” I said, “wow.” We’re doing it since so many years and now it’s working. There’s some room for improvement. Yes. But after so many years, now, it’s working. We also could figure out together with our partners or customers that the a learning benefit is so high — it is so really high — that people can quantify themselves. Motivation is higher. They remember more. But also, the old management guys who aren’t used to work in augmented reality said, “OK, that’s nice, but that’s not a business case because you can never, ever manage to scale this because of the content crash.”

Alan: Yeah. It’s a problem.

Björn: I want to get all this content inside. We also need to get 3D data. Once we get that figured out, you don’t get so many 3D data. Arrows are much more important than 3D data.

Alan: It’s so true. Somebody else came on the show and I can’t remember, but they were saying that at the beginning — it might even be you, I think, when we recorded this previously — you were saying that we used to take a machine and recreate it in three dimensions and overlay it on top of the real machine. And then what people realize is that it’s kind of a pain in the butt, because you couldn’t see the real machine because the digital one was on top. So just take all that away and just put the arrows of what you need. Very simple. “Do this.” An arrow, a finger pointing at it. Seriously, as a technology industry, AR/VR technology, we really overthink things. And sometimes the most simple things are what end up being the most impactful.

Björn: Exactly. And we need to figure this out. So complex to figure this out.

Alan: Yep.

Björn: At the end, we’re only working with arrows and videos. This really helps a lot. And someone needs to generate the content.

Alan: Yep.

Björn: And if you’re having, for example, the trainer who knows how the process works, and then you’re having the guy who knows how the AR works, those two guys play ping-pong.

Alan: Yep.

Björn: “Oh I want to have it like that.” OK. You do it like that. Then the programmer gives it back to trainer to translate. If it’s not what I tried to say, they do it again; they play ping-pong all the time. So in the end, what we’re doing is that we the trainer — the guy who knows what needs to be trained — he can generate everything by himself. But where we need to generate it? Not somewhere in the office. Production’s happening in the production line. It’s the assembly line. So the trainer puts on the glasses, goes into the editing mode, drags and drops what needs to be done. Records basically what he’s doing. It’s pictures, live-recording, using the Hololens or using some other glasses. Let me just say, “Okay, now start the training,” and the training is generated automatically.

Alan: Isn’t that amazing? That’s incredible. So if people are interested in learning more about that, it’s ar-giri.com. Being able to enable trainers to create this content is going to be the key. And I think one more piece of the puzzle is being able to allow companies that create training — because some training, they’re going to recreate environments and 3D models and all this — but being able to have some way for them to share that as well, because the content creation right now is very expensive, it’s time consuming. But I think what we’re going to see is companies that spend a lot of money on developing content, they’re going to want to be able to monetize that as well, because there’s got to be a way to share that content across different entities. And that’ll decrease the time to training right cross the board.

Björn: Exactly. And it’s also important to share the content. Someone could be working with a civilian training company on sharing some 3D model content. Sometimes it’s quite good to test 3D models. Also, you need to think about different training stages of car. For example, the car doesn’t exist; the assembly line doesn’t exist yet. You need to train others in virtual reality, because you don’t have a car. And then you are moving over and the latest car exists, the training is also getting more concrete. What needs to be done? That’s more augmented reality. If you can reuse some 3D models — all those items which already exists — makes the whole process much more scalable. There’s another thing we realized is content creation from what the workers are saying. “It’s nice if you guys generate us a training, but problem is, production is not static. It’s not like you make a product for years and you do it always the same way.” It’s like, if you’re having errors and you’re having errors every week, then they change something, it improves the process. This new process needs, again, to be trained towards the other people. Basically, a training can change every week because production is changing. It’s also not that you’re only assembling one car at one place. You ship different cars; you produce different cars. It’s the same assembly line. And there’s a lot of variation in the trainers all the time. And so for each work place, you need to improve or modify the training all the time, basically.

Alan: Well, Björn, we’re coming to the end of the conversation. But before I let you go, and it has been a fascinating conversation about augmented reality, the different glasses, how companies can roll it out. To package this whole conversation of that, those are the key points. What is one thing, one challenge in this world, one problem in this world that you want to see solved using XR technologies?

Björn: We’re kind of doing the training for the boring jobs. Which is our business, which makes a lot of fun for us. But there are bigger problems in this world, like in education. I feel like my history education was so bad, really so bad. I learned a lot of things later. But augmented reality is such a great tool which is ready to tell stories, stories of our past. Also to tell stories about technology and about complexities about the basics of technology. For example, yesterday in the Deutches Museum they reproduced the labs, the office of Galileo.

Alan: Oh, cool.

Björn: Galileo was the first guy doing structured experiments and filling out a lot of the basic principles. It was looking so nice. And there were a lot of experiments and a few of them explained, but it would have been so nice to have Galileo be inside there. anyway. It would have been stirring and would have so much impact to entertain people so much. It’s the possibility.

Alan: And it’s not hard to do. I’m writing an article on volumetric capture right now. And there’s 55 volumetric capture companies in the world. So far. That we know of.

Björn: And that’s good. Learning and education, you having… in Germany, the best-paid people that the German government pays are teachers. Sorry to say this, but most of them — I’m feeling — are not doing the most wonderful job. There are some good teachers, but most of them, no.

Alan: Well, it’s hard to have the best teachers in a system where a teacher is in one school teaching a group of students. If that teacher now can teach students around the world in a one-to-many VR presentation — or AR — you can now start to bring really contextualized, personalized learning to the world. And I think the world’s education system — systems, because there are multiple systems around the world — but they’re just not going to be adequate for a world where the jobs are changing every few years now.

Björn: Exactly. That’s a different story of what skills that you need in five and 10 years, and 20. But if you’re starting to digitalize, it’s not about replacing old school teachers, but it’s about them giving… teachers need to do different jobs later. And and it’s also about, for example, if you need to teach a certain topic, and there are also different views in the world on this topic. But if, for example, you could take the three best people in the world — you ask a Chinese opinion, a European opinion, and you take the best teacher from the US — who has a certain opinion. You take the best teacher from Europe who has a different opinion. And the teacher from China who is also different. But you would have the chance to record those people once and then giving every student the possibility to experience all those different opinions presented in the best way to give them the best teachers. And then you can get it for the next topic. Also, the three best teachers.

Alan: Björn, I really thank you for joining us and thank everybody for listening. This has been the XR for Business Podcast. You can learn more about Björn and his team at the ar-experts.de. And you can learn more about their training platform, ar-giri.com. Björn, thank you so much.