WGAN Training U: Matterport + Apple Vision Pro: Imagining the Possibilities

WGAN Forum Founder & WGAN-TV Podcast Host 31.21K Website Atlanta, Georgia Linked In Facebook Twitter YouTube TikTok Instragram Matterport Service Partner Google Street View Trusted Photographer Zillow Certified Photographer We Get Around Network Member	31.21K DanSmigrod private msg quote post Address this user	plus 1 plus 1 plus 1 0
	WGAN-TV \| Matterport + Apple Vision Pro: Imagining the 3D/AR/VR/XR/AI Spatial Data Possibilities \| Guests: Hopscotch Interactive CEO & Chief Media Officer Emily Olman (@Hopscotch) and Tosolini Productions Founder and Creative Technologist Paolo Tosolini (@Tosolini) \| Episode #194 \| Thursday, 22 June 2023 Transcript (below) [00:00:02] Dan Smigrod: - Hi all. I'm Dan Smigrod, Founder of the [www.WeGetAroundNetworkForum.com]. Today is Thursday, June 22, 2023, and you're watching WGAN-TV Live at 5. We have an awesome show for you today: Matterport + Apple Vision Pro - Imagine the Spatial Data Possibilities. Our subject matter experts today are Emily Olman, CEO and Chief Media Officer for Greater San Francisco, Bay Area-based Hopscotch Interactive. [www.HopscotchInteractive.com] Hey Emily, thanks for being back on the show. [00:00:43] Emily Olman: Thanks, Dan. I'm super-excited about this topic. Thank you. [00:00:47] Dan Smigrod: Awesome. Also joining us is Paolo Tosolini. Paolo is the Founder and Creative Technologist for Bellevue, Washington-based Tosolini Productions. [www.Tosolini.com] Paolo, thank you too for being back on the show. [00:01:05] Paolo Tosolini: Thank you, Dan. Always a pleasure to be here. [00:01:07] Dan Smigrod: Awesome. Emily for context for today's show, tell us about Hopscotch Interactive. [00:01:17] Emily Olman: Hopscotch Interactive is a real estate marketing agency, as you mentioned, based in the San Francisco Bay Area. Our original tagline was promoting extraordinary spaces through 3D and VR. We've evolved over the years, but we were founded originally as a Matterport and Media Services Agency -- art media services company -- to help companies promote their spaces. I was an early Matterport user and over the last few years, we've evolved. But along the journey, I really dove deeply into VR and XR, and was the president of the VR/AR Association San Francisco chapter. I'm still very active. I'm an advisory council member for Augmented World Expo [AWE] and ambassador for the AUREA Awards, which are in Europa-Park in Germany every year for XR excellence for entertainment. It's very much a part of my DNA now. I consider myself to be a Metaverse Scholar, if you will. [00:02:32] Dan Smigrod: SpatialFirst, 2017? [00:02:35] Emily Olman: Correct. Yes. In 2017 when the Apple ARKit was announced, I joined forces with a couple of other co-founders: one co-Founder in the geo-spatial arena and then another technical co-Founder who had built VR experiences, and we launched a Mobile Interface for spatial computing for commercial real estate. We ran that company from 2017 until 2020. [00:03:05] Dan Smigrod: Called SpatialFirst? [00:03:06] Emily Olman: SpatialFirst, that's right. [00:03:09] Dan Smigrod: Sounds timely and topical for us. [00:03:11] Emily Olman: That's right. [00:03:12] Dan Smigrod: Paolo, for context for today's show, tell us about Tosolini Productions. [00:03:18] Paolo Tosolini: Sure. Tosolini Productions is a digital creative studio. We focus on business storytelling using emerging technologies, which in other words means we help companies -- organizations -- tell their stories by creating interactive presentations using emerging media, including touchscreen solutions for events, trade shows, lobbies, or showrooms. We also do Matterport scans. We have been a MSP [Matterport Service Partner] since 2015. We also have been doing a lot of experimentation and some projects on AR, VR and most recently also AI, just because it all comes together nicely here. [00:04:05] Dan Smigrod: And doing a lot of mash-ups of Matterport with AR, VR, XR, MR, whatever letters they "R" ... ;-) [00:04:13] Paolo Tosolini: That's true. [00:04:14] Dan Smigrod: Tosolini Productions has been doing mash-ups. [00:04:16] Paolo Tosolini: The word mash-up, the definition is really: how can we put together multiple technologies in a creative way, so we can create new value or maybe there is zero value we don't know. But unless you try and we put A + B and maybe + C, then we don't know what can come out. mash-ups are one of our specialties and we love also to share the end results with the community, so we have been doing it on WGAN: We Get Around Network [www.WGANForum.com], on Facebook, Matterport groups and YouTube and so on. [00:04:53] Dan Smigrod: Yes. Thank you for sharing your Matterport mash-ups with the We Get Around Network Forum community. Speaking of mash-ups, what happens when you mash-up Matterport + Apple Vision Pro?Your thoughts? [00:05:07] Paolo Tosolini: You want to go first Emily because then I can start and then we can tag it. [00:05:12] Dan Smigrod: Paolo, why don't you go first? [00:05:14] Emily Olman: Yeah, why don't you go first Paolo. [00:05:17] Paolo Tosolini: It's a good question. I haven't tried the device yet so I can only imagine the possibility here. It's almost like a prompt, a prompt to 'Paolo AI' ;-) imagine what is possible. There are multiple ways that I can think about the Matterport. Do you want to capture data or do you want to consume the data? The processing happens in the middle, so it's less important, in my opinion. But when it comes to capturing, I'm thinking one day and I don't think it's too far away from the date of the release of the Apple Vision Pro that you will be able to get the original Vision Pro, walk around the room and use that LiDAR -- those sensors and scan that room. I bet that apps like Polycam are already salivating with the idea. "I can port my Polycam app into the Vision Pro and it will just work with very few modifications." What do you think, Emily? [00:06:24] Emily Olman: This is why Paolo and I have been friends for so long. Because we are birds of a feather, we are both huge fans obviously of the Matterport system and also of Polycam, which was my favorite app that I was playing with in 2022: hands down. If people haven't used that before, Polycam uses your mobile LiDAR on your device and helps you scan objects and spaces. You can scan a room, you can make floor plans. It is a very elegant way of capturing space. We've always said in the VR community, I think that Paolo is one of the leaders in this because he really treats this like a community which was one of the things that attracted me to the VR space was that it was a group of like-minded thinkers and people that wanted to experiment. The joy that brought us meant we were happy to share what we were doing with other people. It's about learning because we all knew there was no business case that would sustain the whole industry yet. [00:07:36] Dan Smigrod: Paolo has proposed that the Matterport Capture app with the Apple Vision Pro might be a reality. Apple came out with the VisionOS SDK [Wednesday, 21 June 2023]. Do you think Matterport is already on it in terms of capture with Matterport + Apple Vision Pro? [00:08:00] Paolo Tosolini: Jumping in Emily, I have an opinion there. [00:08:03] Emily Olman: I think it's true. The reason why though, and you have to understand is that if you look back at all other previous headsets, they were scanning the geometry and they have LiDAR but they're not combining photography and LiDAR. That's the difference. Apple Vision Pro is a 3D camera plus the object recognition or scanning ability, LiDAR ability, and then the Apple M2 processing chip. It has everything under the hood that it needs to be able to create a Matterport scan and that's new. That's new for a headset. Yes, I agree with Paolo 100%. [00:08:46] Dan Smigrod: Do you think that's going to happen, Apple or excuse me, Matterport is going to release the Matterport Capture app via the Apple Vision Pro? [00:08:58] Emily Olman: Well, I don't know why they wouldn't. [00:09:02] Paolo Tosolini: I'm not sure if it would make sense, personally, to release it. They can easily do it, but imagine you are scanning a 360 by doing this 100 times in a house. I don't think it's practical and it doesn't make sense. Technically it's feasible? Is it practical? I don't know. [00:09:25] Dan Smigrod: Let's go into things that might be practical. I like to think of practical things that: 1) make money, 2) save money and 3) save time. In the category of making money, what might be some of the use cases of Matterport + Apple Vision Pro? Emily. [00:09:44] Emily Olman: Probably you have to think of Matterport evolving into a wayfinding and augmented reality [AR] platform more than the current version of Matterport, which is really about the dollhouse view and tours. You have to look back at maybe even the timeline that you and I discussed [WGAN.INFO/mptimeline] where Matterport switched to a digital twin company. After people get excited about spatial computing again, all of a sudden Digital Twin, I promise you will be the next WGAN-TV Live at 5 we're going to want to do because everybody is going to want to talk about Digital Twins. Yes, annotating spaces with information and Matterport having a lot of third-party developers [WGAN.INFO/mppartners] that are doing that and also Matterport bringing that into its native software, I think that's where you're going to see it more than like what Paolo says doing 360s and a headset, not as interesting as wayfinding, labeling, documenting, all of these other things, Matterport is also good for it, but will be optimized on a headset. [00:10:55] Dan Smigrod: Emily, you've done some mash-ups of augmented reality [AR] meets Matterport using [an iPhone] paired with a Matterport Partner's tech. Will the Apple Vision Pro be a better way to experience wayfinding than just using your smartphone? [00:11:19] Emily Olman: Well, from a form factor. I think it's a good question because right now, Apple Vision Pro seems to be built for more stationary experiences. I don't know what it's going to do either. For example, moving indoors to outdoors. That's one of the questions that I have. Again, I'm like Paolo, I haven't tried it yet. But right now, even an iPad can work if it has the tracking or a phone, but I'm not so sure that I'm going to walk around with the Apple Vision Pro the same way I would maybe with the Magic Leap or with a different headset. [00:12:06] Dan Smigrod: You mentioned a couple of other technologies. I wrote down a list before the show: HTC Vive, Meta Quest (Oculus), Microsoft HoloLens, Magic Leap, Android ARCore, WebXR. Is there anything about the Apple Vision Pro that is different from things that have previously been done? [00:12:38] Emily Olman: Paolo, do you want to jump in? [00:12:39] Paolo Tosolini: Sure. I think yes, there are differences and I think everything that has happened before, it helped Apple to come up with the current device. Magic Leap that I think, Emily, you might have one right there. [00:12:59] Emily Olman: Yeah. Shall I demonstrate my Magic Leap guys? [00:13:06] Paolo Tosolini: Yeah. I used to have one too. [00:13:09] Emily Olman: I will put it on. Okay. Even with my ponytail, it still works. Here we go. Remember the battery pack here. That'd be another thing. [00:13:19] Dan Smigrod: But I want to say. These things didn't really take off like a rocket ship that perhaps early adopters thought would happen. Is there something different about Matterport + Apple Vision Pro than maybe some of the preceding VR, AR tech? [00:13:41] Paolo Tosolini: Yes, I think the technology has matured. We went through some of the summers and winters of VR. I think people are more used to these devices, not to the extent that everybody has one necessarily, but people are more accepting. Along the way Microsoft came out with HoloLens instead: two versions of HoloLens. I don't know if they're going to come up with a third, who knows? But every first version of any new hardware is difficult. You have to figure out how to sell it and the use cases, and so on. At the beginning of HoloLens they didn't even know who to sell to. But now we have the US Army. We have a use case for remote assistance, remote help, and architectural visualization. Those are becoming more enterprise. I believe, and confirm for me, Emilly, but even Magic Leap is thinking, these devices cost a lot of money, they're difficult to build, let's sell it to the enterprise that they have more money, and because $3,000, so how much it costs now. [00:15:04] Dan Smigrod: Starting at 3,500 for the Apple Vision Pro. I wear glasses, so presumably it would cost me more for some Zeiss lenses for the Apple Vision Pro. I'm trying to understand -- [00:15:18] Paolo Tosolini: To me it's an evolution. But until you put this headset into the hands of 100,000 developers, it's hard to figure out all of the different and vertical use cases. In my opinion, when we think about Matterport and Apple Vision Pro, I see a world of visualization that takes it to the next level. Again, it's not about scanning, it plays, it's about visualizing it. All this data coming is in the digital twin. I can grab the digital twin in front of me. I can call somebody -- can call Dan and Emily -- have a FaceTime with them, and say, "what do you think about this building? Let me rotate it. Then I see you pointing to a live data source and you click and then you zoom-in from the dollhouse into the actual walk-through. To me, that's a strong use case. [00:16:14] Dan Smigrod: Let me pause there and ask you a follow-up on that. I'm going to assume, correct me if you think I'm totally wrong, but that every existing Matterport tour should work fine within an Apple Vision Pro. Let us make that assumption. [00:16:31] Emily Olman: It does. [00:16:32] Paolo Tosolini: It does [00:16:32] Emily Olman: It already would. It's possible. [00:16:35] Paolo Tosolini: Yes. It's possible. [00:16:37] Dan Smigrod: Paolo, you mentioned two things: using Apple FaceTime to communicate and collaborate around a space. Do you think that's a significant use case of a mash-up of Matterport + Apple Vision Pro + Apple FaceTime (or some other mechanism for communicating around a space -- collaborating, and communicating)? [00:17:09] Paolo Tosolini: Yes. When it was announced, I had to watch twice -- the Apple keynote at WWDC23 -- just to get it and to understand it well. Then I went to the Apple developer sessions and started watching the individual sessions. Apple is putting an emphasis on collaboration. They have these frameworks, frameworks to share things. Frameworks to design things and sound. Now sharing content in visual -- staying in the same space -- and connecting on the same object, for example, I think it's a pretty strong use case. Has it been explored enough? No. Because everything is so new. But, it's easy to think how we can have a FaceTime and we can collaborate on a 3D model. The 3D model is doable, by the way, I think [Wednesday, 21 June 2023] with the VisionOS SDK, Apple also announced that they are supporting WebXR Standards. Emily, I think that's what I heard. [00:18:22] Emily Olman: Yeah. [00:18:23] Paolo Tosolini: Which means that you open Safari and potentially you could have 3D objects right there which is pretty cool in FaceTime. Then you and I, we can all talk about the same object and picking up and moving to a different room. We don't know what we don't know, but I think the vision is there. [00:18:46] Emily Olman: Those are cool things, Dan. Also, you asked about what's different. One of the things that's really different with this design is the video pass through. You have to imagine when you have a Meta Quest 2, or you have one of these headsets on -- or a VR headset, for example. This passthrough camera was a really important step which enabled us to have -- "I can exit my experience and I can see the world around me." What's really innovative, and apparently from the reports I've heard, so groundbreaking about the Apple Vision Pro is that the pass-through is the highest fidelity, it's the most lifelike. You really are not feeling like you take your headset off and you just left this other experience and then you came back to the real-world. What I like about it is that maybe playing off of what Paolo is discussing on the collaboration is that instead of feeling you're going to collaborate and you have to enter into a game environment. You'd have to enter into this other world in order to do it. The collaboration feels like it's just an extension of your desktop. It feels like it's an extension of our reality because it is powerful and the pass-through is so much better. If you add to that one other thing, well, I can get into it later, but you're using gestures to control the device right now. You're not using a controller so that would be the question that I would have as a user, does Matterport's virtual tour content lend itself to gesture control? Or do we feel like for the precision of collaboration, we will need some equivalent of a controller or something else. I have a lot of questions, but I think it's very different from what we've seen so far. [00:20:46] Dan Smigrod: Sorry, Paolo go ahead. [00:20:48] Paolo Tosolini: No. I was just piggybacking on what Emily said. Matterport support will be out-of-the-box: open Safari. You open a Matterport link. You will see your model. Now, is this cool enough to purchase an Apple Vision Pro? I don't think so. However, I could see Matterport trying to think, "what else can we offer to make it even more immersive?" [00:21:19] Emily Olman: That's it. [00:21:21] Paolo Tosolini: We don't know yet what that will be. Is it about the 3D model? [00:21:24] Dan Smigrod: Well, that's why we're asking you. ;-) What is it? What might be the possibility of instead of -- how to say this? You take Matterport, you stick it inside of Apple Vision Pro. That doesn't really seem to take advantage of every technology that's now going on in Apple Vision Pro. If you have 12 cameras, five sensors, six microphones. Emily mentioned gestures, tapping, flicking, expanding, voice, tapping up here, controlled by eye. What are the possibilities? Instead of having these things, devices on your wrist where you can just tap or look or touch or stretch or shrink? Is there something exciting where Matterport gets mashed up with Apple Vision Pro? [00:22:21] Paolo Tosolini: Do you remember when Matterport supported -- was it called [Matterport Core VR]? [00:22:27] Emily Olman: Yeah, [WGAN.INFO/mpcorevr] [00:22:28] Paolo Tosolini: It felt more immersive. People already said, "Oh, this is so cool. However, I can only move, but I cannot have all the Matterport MatterTags and so on." What else could be done if you can blend the environment in Apple Vision Pro? I'm just brainstorming right now. But imagine that with the little wheel, you can switch in-between the pass-through and Matterport tour. You are in the location. Then you put that MatterTags on the real space. Now you turn on the MatterTags and then you see the MatterTags, but you see the actual room like here. I tap the MatterTag and I see magically things happen. But all those MatterTags are being added originally on the Matterport platform -- and they perfectly overlap and overlay with the actual environment. To me, that's pretty cool. [00:23:30] Dan Smigrod: - Yes. But going back to -- Emily has done these mash-ups. I think, in fact you have too, Paolo, of Matterport + AR + a smartphone, is an example. I'm just wondering, is there something different and unique about the Apple Vision Pro mashed up with Matterport that is a better -- I don't want to say viewing experience because maybe that's not the right thing, it's some interactive experience? [00:23:57] Emily Olman: - I have an idea. When you look at the current generation of how Matterport has gone. For example, originally we had to have the Samsung Gear VR, we had to have those Matterport spaces basically handcrafted by the Matterport VR team. [00:24:21] Dan Smigrod: - You're going back in time a little bit. I think it's about 2015, 2016. [00:24:22] Emily Olman: - I go back in time. [00:24:24] Dan Smigrod: - 2015, and Matterport offered VR in Samsung Gear VR and Google Cardboard. I want to say stereoscopic virtual reality experience, but it could only be created by hand, and you paid a lot of money to Matterport. [00:24:49] Emily Olman: - You paid about $500 a space and you had the Matterport VR Showcase app. You still had an app. You had a Matterport VR Showcase app and that was how you accessed it, and then you would put it in your Gear VR, then later we move from that into like, let's just jump a little bit further forward. We had the [Matterport Core VR] where they got rid of the really nice transitions. Then we went from pano-to-pano. [00:25:15] Dan Smigrod: - Just to clarify for our audience. The original experience. I would say -- it literally felt like you were walking through the space. The next-generation was [backwards] really moving from bubble-to-bubble. It didn't cost you $500 and it was available, I want to say immediately, but it wasn't the same VR experience. I know my wife from that first-generation, she put the Gear VR on for the first time, got nausea and said, "No. I'm not going to use this again." Then maybe the second-generation was designed to reduce that nausea, but it never went anywhere. I didn't think of VR going anywhere with Matterport at that time. [00:26:01] Emily Olman: - It was definitely not a priority to them because they were trying to scale other things. It was one of the VR winters, if you will. But what they did instead of really supporting that, is that, they did make a change a couple of years ago where they introduced a different code in their links. If you go on now to copy a link in the Matterport system, you'll see that there is a bit of code that says xr-spatial-tracking. When you have xr-spatial-tracking on your links, then if you have your model on your website -- right now today, any browser -- and you use that current link, and you put the model on there. If I go in and I navigate to your website, you can go to www.HopscotchInteractive.com now if you want in a Meta Quest 2 VR headset to or any of these. Today, you can view any Matterport space, it will sit in your VR headset in this Meta Quest 2, it will say, "Do you want to enter in VR?" Enter in VR: right from the browser. We already have this. So your question a few minutes ago said, "are people going to be like, this is the thing that's going to put me into Apple Vision Pro?" Probably not because we already have it and it's not a big game changer. The difference is that [the Meta Quest 2 VR headset] is more or less a gaming system. This is really about gaming. It's not about enterprise use as much. [00:27:32] Dan Smigrod: - You're talking about the device in your hand rather than in application? [00:27:36] Emily Olman: - Exactly the Meta Quest 2. [00:27:38] Dan Smigrod: - Let me just try and ask that question again. If we're talking about a virtual reality experience, a VR experience meets Matterport. That seems like it fell flat. It fizzled. You called it a VR winter. Paolo called it a VR winter. How important is it that Apple has come out --It says that the Apple M2 chip + the Apple R1 chip to make a latency free viewing experience. The 64 pixels for every pixel of an iPhone, meaning there's 23 million pixels on a postage stamp size screen in each eye, or the equivalent of 4k each eye. I remember when I was using the Samsung Gear VR with my Samsung phone, it looked like "burlap" in terms of the screen resolution. It was of course, and it sounds like apples about to come out with a silky smooth viewing experience as you described it, Emily. Also the fact that you can come in and out of the experience of what I think of being in the real-world versus immersed in a virtual world. Are these factors super-important in terms of the potential success of Matterport + Apple Vision Pro? [00:29:01] Emily Olman: - Go ahead, Paolo. [00:29:06] Paolo Tosolini: - I'm thinking that the in general for the success of the Apple Vision Pro, having a 4k resolution for each eye is very important because it opens up the possibility to do actual work: to read text; to spend more time on the device and do something meaningful other than just gaming. Gaming, if it's low resolution, I can bear with it, but I cannot do actual work on a low resolution screen. Now, how will Matterport benefit from the combination of this hi-res? Well, photography will be better. The tour will look better. However, I'm thinking of something else. Matterport as a company is hosting all of these virtual tours and so on, but it's becoming a data company. They just released a week ago, Matterport Genesis AI [WGAN.INFO/mpgenesisai]. [00:30:10] Dan Smigrod: - June 14, 2023 -- Matterport Genesis AI -- like ChatGPT4 for inside of a Matterport tour. Isn't that crazy and exciting? Relevant to our conversation for Matterport? [00:30:23] Paolo Tosolini: - I think so. [00:30:25] Dan Smigrod: Matterport tours + Matterport Genesis generative AI + Apple Vision Pro? [00:30:32] Paolo Tosolini: AI, these keywords, perfect. ;-) [00:30:37] Dan Smigrod: Just to pause for our viewers, that means AI. You're looking at a house, and you type in, "show me the house without any furniture." Boom! It's clutter-free. Type in, "show me the house with contemporary furniture." Boom! It's filled in. How significant is Matterport Genesis generative AI in a Matterport tour + Apple Vision Pro experience, Paolo? [00:30:55] Paolo Tosolini: To me, it is significant. For example, a simple use case. I do a Matterport scan of an empty office. I load it into Apple Vision Pro. Then, through Matterport Genesis generative AI, I'm standing in the middle of this office and I can pre-visualize what this office might look like if we subdivided this open space or in individual rooms. This is what Emily already did with her app. [00:31:28] Dan Smigrod: Yes. Is this better than holding up an Apple iPhone or an Apple iPad? [00:31:33] Paolo Tosolini: Totally. [00:31:33] Emily Olman: It's going to be better, but also, it's not as universal yet. That's one of its problems is that the price point is intended right now for developers and for early adopters. And it's going to show you what the future will be. Yeah, Paolo is right. This is totally relevant. Matterport Genesis AI -- being able to have Matterport combined with AI and to be able to with a verbal prompt, if you will, pull a model off of a URL, pull it out, grab the model and bring it into wherever you want it to be. You can put it around you, or you can hold it and probably rotate it. Paolo and I could be standing around it and looking at it, like we already do with other AR experiences. [00:32:23] Dan Smigrod: So let me try a scenario and then have you comment on it. Emily comes in, and she scans a 30,000 square-foot empty commercial space. There's no drywall, nothing. And the space is available for purchase or lease. Wherever it is. Okay, now you've scanned it and then, you are in your Matterport tour [of a 30,000 SQ FT empty commercial space] with an Apple Vision Pro -- perhaps even now standing back in the space that you were -- and say to Matterport Genesis AI, "how many people can you fit in this commercial space?" And then all of a sudden, Matterport Genesis AI starts populating office furniture within the space. Now, Paolo calls on Apple FaceTime and says to Emily, "hey, I want to show you what the space looks like configured with the most number of tables. Then Emily says something like, "show me with a more collaborative space." It reduces the number of tables. It reorganizes the space. Is that important? Paolo, does that blow you away? [00:33:35] Paolo Tosolini: To me, it's a use case, especially for commercial real estate. Is it 50% of use cases? No. It might be the 10%, or I don't know, for commercial real estate. [00:33:47] Dan Smigrod: I'm sorry. Did you say edge-use case? [00:33:49] Paolo Tosolini: No. I'm just saying, "is it like the use case that will make Matterport and Apple Vision Pro sell a lot?" Probably not. But for commercial real estate. I think it's pretty cool. [00:34:04] Dan Smigrod: Yes. Because I always think that innovation will always take place at the intersection of Tosolini Productions + Matterport + Apple Vision Pro + a use case that's about making money. ;-) So I'm always thinking that's the first use case. [00:34:22] Paolo Tosolini: Making money. ;-) [00:34:23] Dan Smigrod: Is making money; it's sales and marketing. So if a sales and marketing person could lease out that 30,000 square feet faster for more money, you don't need to have a bazillion people look at it. You need to have the right person looking at that space -- [00:34:40] Paolo Tosolini: I agree. [00:34:41] Dan Smigrod: -- who couldn't visualize how that space could look? Is that -- Emily? [00:34:48] Emily Olman: You're killing me, Dan. ;-) Well, I feel like I need to send around the SpatialFirst pitch deck to everyone and bring up the slide that we called the PlaceTime. We built PlaceTime Oakland [WGAN.INFO/PlaceTime1], which was basically using the SpatialFirst app, and we got the grant from Magic Leap to port it to the Magic Leap because we saw that this was where it would head. Unfortunately, with the pandemic, we didn't quite get the chance to see it all the way to completion. Things have changed. The thing that we always said at the beginning of that pitch for SpatialFirst was that an empty space is painful. There's a lot of empty space right now. There's a lot of people not going back into commercial office space. It's still just as hard to help people visualize something. [00:35:43] Dan Smigrod: Emily, you're in the greater San Francisco Bay Area. I believe the commercial real estate vacancy rate is about 25% today. [00:35:51] Emily Olman: That's correct, 25%-30%, and also particularly heavy in places like our central business district, the downtown financial district. We feel it and I think that this is the thing. In the current scenario, getting visualizations quickly and bringing the cost almost to zero. In a way, if you're able to use Matterport Genesis AI to give a verbal prompt to revise the vision of a space, almost immediately, assuming you have low latency, and you have a good stable Wi-Fi connection, then you are there in real-time, revising and collaborating and visualizing and having a shared experience with people who may not be in this space with you. That's what PlaceTime - that whole concept was trying to achieve. [00:36:52] Dan Smigrod: How important is it to be standing in that empty space and doing the Matterport + Apple Vision Pro? Is that part of it is having the client take off the Apple Vision Pro at some point or no, they're looking right through it. They can see an overlay of the space. [00:37:13] Emily Olman: But if they need to roll into it, they need to use that Apple Vision Pro button to basically turn it into a VR experience. They're able to have the ability to move through the space while they're stationary, like you do in VR now, and they don't need to be in the space. So you could either be in the space and see the augmentations like these points of interest and these different data points can be in the space, or you could do that remotely. It's much more comfortable, obviously, to not have to leave your house. But if you need to go somewhere to view a space, then you want to be able to impress somebody as much as you can when you're there. It is important, Dan, it's really important. I think that owners/operators of buildings are even more struggling now to figure out how to stand out. [00:38:13] Dan Smigrod: Let me switch the conversation to a related space. You're in that commercial office building and elsewhere in that building is a Mechanical, Electrical and Plumbing space (MEP space). I imagine that there's a lot of training that needs to take place around this MEP space. We've talked a little bit about making money, but I think saving money and saving time; I'm going to guess training is a big category for that, Paolo, I see you're nodding your head. What would you like to talk about on this topic? [00:38:43] Paolo Tosolini: I was going to actually bring this up. Real Estate might be one of the scenarios. But imagine if you are in an industrial setting. You need to control in real-time certain values, and those values are not there. There is no display there, but the display is virtual. So you put it down suddenly through the Internet of Things (IoT) and you can see green, yellow, red, the things that need your attention. [00:39:15] Paolo Tosolini: Remote Assistance is the other use case that the Microsoft HoloLens jumped on because there is money to be saved. Then the technicians go into this control room; they find something that is broken or is malfunctioning, they need help because they don't know exactly how to repair it. They call home and in theory, via FaceTime somebody is helping the technician figuring things out. Training could be another scenario like a simulation. [00:39:48] Dan Smigrod: Excuse me, which leverages what you've talked about earlier in terms of -- communication and collaboration meets training; meets Matterport; meets Apple Vision Pro; meets some overlay of IoT and AR. [00:40:00] Paolo Tosolini: Exactly. Whatever the user sees, it can be seen by somebody else, an engineer at the headquarters. [00:40:10] Dan Smigrod: I still want to ask the question though. It just begs the question. I listed out all these devices that exist today that sound like exactly what we're talking about. What's different about the Apple Vision Pro versus all the VR and AR headsets that exist today? [00:40:29] Paolo Tosolini: To me, the field of vision I think is important. Right now with Microsoft HoloLens it was HoloLens 1 it was this big, HoloLens 2 is this big, I think is the same for Magic Leap. Now we're seeing a lot. We feel we are 'there' and I think that makes a difference. Whatever is the graphic, the 3D model that comes out, it's very high resolution, it feels real. It stays there, it's locked because all of these sensors understand where this object is positioned and it doesn't move. To me, that sense of reality has a lot of value. Emily, what do you think? [00:41:18] Emily Olman: I think that I want to look at the use cases. For example, if we look at the Microsoft HoloLens which has the advantage and it has always had the advantage of having existing relationships with the enterprise. It was a natural fit for Microsoft HoloLens to be something that a salesperson who's already repping Microsoft can go and say to their existing billion-dollar client, "hey, let's roll out a test with these HoloLenses. This works with Microsoft 365 suite of apps and your suite of services." Remember Microsoft I think has this partnership now with Meta also to bring Avatars in. There's more to come from the other players but our focus right now is Apple because it's such a big deal. But RealWear has a very rugged device that is made specifically for bringing up information and remote assistance in the field. They have a RealWear 2.0, amazing device. I just demoed it at AWE and it's really specifically made to be thrown on the ground, or with a hard hat, worn with a hard hat. Now, my sense of the Apple Vision Pro is that the use cases that they showed were ones like somebody sitting on their couch, or somebody in an office space, maybe an architecture firm, or somewhere where it's not as much of a rugged environment. The memes I saw day one were: every Apple Vision Pro person the next day and it's showing like a cracked screen. Did you guys see that one? That's our experience of Apple products, is that, "oh my gosh, what if I drop and then the screen breaks? There goes $3,500." It's going to need to be less delicate if it's going to be used in those more extreme environments. Because it's a hot device, it has a lithium battery. It's going to have to be rated to be used in environmentally hazardous areas. It's not an automatic fit for AEC or what did you call it? MEP? [00:43:44] Dan Smigrod: Mechanical Electrical and Plumbing (MEP). [00:43:46] Emily Olman: MEP. Is it a natural fit for those environments? I would say my first guess is no and then it's going to instead it will rely on the fact that it has the world's largest ecosystem of apps that you use for personal, for photography, for sharing and communication and maybe collaboration. [00:44:07] Dan Smigrod: But let's bring it back to Matterport because that's our audience. If you're a Matterport Service Provider, is this something to get excited about because you see this is a new business opportunity with existing clients or potential clients? Let me ask the question differently. Paolo, you work with a lot of very large clients. Are you in stealth mode -- as soon as we hang up here -- working on building out a Matterport + Apple Vision Pro story for your clients? [00:44:44] Paolo Tosolini: It's a hard question. I don't have a definite answer saying there is one scenario that will be the use case for everybody. To me, there's going to be a lot of experimentation. Experimentation that will go well, some other experimentation there will be extreme and useless. Until we create those mash-ups and we try them out and we figure out -- this is not necessarily, this goes nowhere -- I think it's hard to predict what will stick and what will not. [00:45:18] Dan Smigrod: Do you have a corporate client in mind that you're thinking, "oh, I have the client? ..." [00:45:25] Paolo Tosolini: To me, the low-hanging fruit in my business is more like, how can we visualize information or make an impact about the story in a spatial setting? If we can bring in Matterport as is, from Day 1 there's going to be just a Matterport virtual tour. We can go to the extreme. You can extract the 3D model. But even if we extract the OBJ, the Matterport MatterPak and so on, when we look at it in high resolution or with all those defects, it will look worse than what it is on a screen. Maybe we don't want to go down that path. Maybe the key is using Apple APIs to scan a floor plan and then suddenly we can mash it up with Matterport somehow. It's hard to predict the possibilities. From Day 1, a lot of the iOS apps will be available, some of them will be relevant, some others will not -- [00:46:30] Dan Smigrod: Me personally I was just totally disappointed. Because I don't think looking at Word, or Excel, or something bring it into -- [00:46:40] Dan Smigrod: I can't visualize using those apps in the Apple Pro Vision experience. [00:46:43] Paolo Tosolini: Dan, I know, but we don't know what the real clients might want. They may need an Excel spreadsheet on one end, that they click a link and they see Matterport on the others, and that for them is the killer app. For us, we're dreaming about all this could be. [00:47:00] Dan Smigrod: Expand on that. What might that be? What does that table do in relation to Matterport, for example? [00:47:06] Paolo Tosolini: I'm just making this up right now, but imagine you have a spreadsheet with all locations, and then you want to see what this location might look like. You just have a Matterport link, you click it, you get a Safari, and then you start going into immersive mode and suddenly you're navigating the house, and then you come back into reality, which is your visual space, and you go back to the spreadsheet. This cannot be done on the screen, on a Mac or, on a PC. You have to start switching devices. Even if you switch devices, you might not have the same experience. To me, this is a cool low-hanging fruit scenario. Is it super-cool? [00:47:58] Emily Olman: Can I jump in here? [Please!] I want to go back to, actually, this is part of the thing that is so interesting for me is that, we are not going to have to reinvent the wheel, we just have to remember what we already did five years ago. For example, I want to share that Matterport had something called the XR Collection or VR collection. Basically what it was -- and probably me and maybe five people used it -- I have no idea, but, Dan, hold up your little Google Cardboard again. [00:48:36] Dan Smigrod: I think Paolo has one too. [00:48:38] Emily Olman: Does it have a QR code on the side? Do any of yours have QR codes on the side? [00:48:42] Dan Smigrod: Been so long since I used it, I just don't even remember. [00:48:44] Emily Olman: Paolo's has a QR code on the side. Now, imagine this. The thing is that the way we used to do it was we would help the clients. They would order a Google Cardboard, they would put a sticker on with a QR code, they would scan it with their phone, and they would pull up a Matterport VR collection of spaces that were already preloaded for them to view. It was just a really slick thing where I would scan all the spaces and then they would go into that collection and Matterport would help make the VR collection. I think I had to go through Matterport Support to get it. There was a web URL that I had, and I could fill in the spaces on my Matterport models, and then basically, I got one link, and that one link I just connected to a QR code. Imagine it's basically that same concept applies with the Apple Vision Pro, and instead of having to go through QR codes, scan the link, do this, instead, it's a webpage that has not what Paolo is suggesting with a list and then the links to the models. It's the models themselves that are there. [00:49:51] Dan Smigrod: Yes. [00:49:53] Emily Olman: Nicely organized. [00:49:54] Dan Smigrod: Forgive me. How important is the ability to tap or look, or pinch in order to go to that Matterport space? Is that interface super-important because it just made it transparent or magical to be able to select one of those 10 spaces you're talking about, Emily? [00:50:13] Emily Olman: Sure. Do you want to read it or do you want to just look at it and go into it? It's whatever you do to reduce the friction between the time it takes to get the information, and then providing that display of information. For sure, it's better. It's faster, it's more intuitive. [00:50:35] Paolo Tosolini: It's convenient. [00:50:35] Dan Smigrod: Let me go into a little bit of a lightning round of some of the features. I know Paolo, you talked about FaceTime and the 3D camera, 3D photos, 3D video, seeing each other. Is the battery in a pocket that gives mobility, is that important? Not important? [00:51:03] Paolo Tosolini: Well, I think it's a necessity. If you want to take it someplace and then have the power for two hours, you need a battery pack. And then the processor is so powerful that it needs a battery. [00:51:16] Dan Smigrod: Visual search. The ability to look something up, visually? [00:51:22] Emily Olman: Cannot be underestimated. It's already being used in Google Lens or Google so much. [00:51:33] Dan Smigrod: Can you give us an example, Emily? [00:51:35] Emily Olman: For example, let's see. I could just reach for anything or something that you don't know... [00:51:41] Dan Smigrod: I'm trying to relate it back to a Matterport tour though, so what is it? It's something that I'm looking at within a Matterport tour? [00:51:49] Emily Olman: Take a snapshot in the Apple Vision Pro of an Matterport image and run that through a recognition type of an engine, those kinds of things, where it would recognize all the objects and it would tell you where to buy them. It could give you links to go visit the websites to buy them. That's the thing, is that it becomes fluid with all the rest of e-commerce. [00:52:12] Dan Smigrod: Paolo, why are you shaking your head up and down for visual search? [00:52:17] Paolo Tosolini: I was just visualizing what Emily was saying. Imagine I navigate into a living room, I see a beautiful sofa. Imagine if by just looking at it and by tapping. The Google Lens, that's the same thing, so it's just a reverse search through an image that is being captured from within Matterport and suddenly it is populated with all sorts of e-commerce. Pretty powerful. [00:52:48] Dan Smigrod: You're excited about that!? [00:52:50] Paolo Tosolini: It's pretty cool. [00:52:52] Emily Olman: It's not just Matterport, Dan, though. Matterport is one way, but there's many other ways. [00:53:00] Dan Smigrod: I just want to try and stay focused on Matterport + Apple Vision Pro. [00:53:01] Paolo Tosolini: Dan, this could be an Add On to the existing. -- We probably don't need -- even to just be looking. It could be an Add On to one of these overlays. You just right-click, you select the image and suddenly it just goes and does the reverse search. [00:53:23] Dan Smigrod: You might even just look at it. Maybe Matterport can figure out to say, with a voice command or eyesight that you just added that ChatGPT-4 functionality to say, "what are my other sofa possibilities right here from IKEA?" [00:53:41] Paolo Tosolini: Let me tell you another thing. My wife, she's an interior designer, stager and so on. [00:53:47] Dan Smigrod: Francesca, yes. [00:53:48] Paolo Tosolini: Yeah, Francesca. Imagine if somebody goes into a room, scanned with Matterport and then can consult in FaceTime with somebody like my wife, who can say, "you know what, I think we should rechange the setting of this room, and here's how I suggest doing it." Because all of these assets are populated through Matterport Genesis AI, they can be moved. They're not stuck there. And suddenly the arrangement is being done in real-time through FaceTime, right in front of your eyes, and then you feel you are there and you see all this furniture moving and it says, "I like it. I see what it looks like." [00:54:40] Dan Smigrod: That might even create more opportunities for a specialist who doesn't need to travel on-site and can only go visit two or three sites a day, now is available on-demand. When you're asking Matterport Genesis AI -- this ChatGPT-4 meets Matterport space --that rearranges the furniture. You say, okay, "let me ask a real designer like Francesca Tosolini," and say, "what do you think? How might we improve on this?" [00:55:11] Paolo Tosolini: Now, I don't know when Matterport Genesis AI will be released. Do you have an idea? [00:55:14] Dan Smigrod: [Matterport's June 14, 2023 media release said, "early releases expected before the end of 2023.] [00:55:24] Paolo Tosolini: It seems to me almost we need to be there with generative AI. Let's announce something, but it's a little bit early on. [00:55:34] Dan Smigrod: My lightning round is going very slowly here. Let me try this with a different topic. Matterport + Apple Vision Pro + Unity. Does that do anything for you? Emily? [00:55:48] Emily Olman: I mean, you can make it more into a game experience. You could go that direction. I think of Unity as something that enables more game-like play, moving the camera around, I don't know. It could exist. [00:56:07] Dan Smigrod: Paolo. [00:56:08] Paolo Tosolini: You can import assets from Matterport or you can create assets in Unity, and then the two can blend together and create some game, like Emily said. [00:56:18] Dan Smigrod: Sure. Apple spends some time talking about spatial audio, and to accomplish that is doing 3D mapping of the actual room. Any comments on that Matterport + Apple Vision Pro + 3D mapping of a room in real-time? Is it an AR overlay? [00:56:43] Paolo Tosolini: I'm just making up a scenario here. Imagine that if somebody who sells audio equipment, they want to pre-visualize what the room and what the outcome might sound like, and what different effects might have been purchased in different equipment using visually, positioning differently of the speakers and sound, that might be a sale. I don't know. Is it essential? I don't know. That's cool. Sure. [00:57:27] Dan Smigrod: I believe Paolo, you mentioned that you watched the live stream of the WWDC23 Apple Keynote. Let's call it twice. I watched, I guess I was one of one million. I then watched the Apple Vision Pro10-minute video twice; 51 million people [as of June 22, 2023] have watched that. Sounds like there's some interest in an Apple Vision Pro. Emily, are you going to buy one, day one? [00:57:56] Emily Olman: Yeah, I was hoping there would be a pre-order at the end of the Apple WWDC23 Keynote, but unfortunately, there wasn't, so we have to wait until 2024, but I actually already know people who have one, and so I'm hoping to be able to get a hands-on at some point in the San Francisco Bay Area, I have the advantage of having local friends who are already deep in the weeds on it and we're invited to the WWDC23. [00:58:24] Dan Smigrod: These are friends and colleagues that are building third-party apps for the Apple Vision Pro ecosystem? [00:58:32] Emily Olman: Correct, and so they're already things that have been available for VR headsets already, and now there will be building a port to the VisionOS. [00:58:46] Dan Smigrod: VisionOS. Are you buying it as soon as you possibly can, and you are in? [00:58:52] Emily Olman: Yeah. I mean, I don't know why I wouldn't. [00:58:55] Dan Smigrod: Is that a learning experience? Do you have something specific that you want to accomplish? You just like, "well, of course, this is the future arriving and you have to be part of it?" [00:59:06] Emily Olman: I'm not thinking about it like a desktop, Dan. I'm thinking about it like a camera. It's not a desktop for me, it's a camera. It's a new camera and it's a new way to take photographs, and it's a new way to communicate and to create media. For me, even if it were all of these things that it is right now, I think of it as the future of photography and I think that was mentioned. [00:59:37] Dan Smigrod: But we also don't know whether that's taking an eight-second 3D video clip or a minute 3D video clip or that you can shoot longer than that, but you don't care. It's the next generation of photography. It's not related to Matterport, but you're getting one soon as it's available? [00:59:56] Emily Olman: Yes. [00:59:57] Dan Smigrod: Paolo. [00:59:58] Paolo Tosolini: Yes. I'll do the same for multiple reasons. One for research purposes. If you know that you are in this field, why wait a year? Knowing that this device will have a lifespan, probably at least two years. I don't think Apple will come up next year with another device. Why not be in the game sooner? Learn as much as we can and experiment and be early on to the market and pitching to potential clients as a service, as an Add On, who knows? Yes, I'll purchase it as soon as it's available. You can always resell it. If it doesn't go, I bought a bunch of devices, I sold them all, so it's like holding on to all the equipment. It doesn't necessarily make sense to me, at least monetarily. But this is new and we have to be in. It's something so new that is so important that you don't want to wait for the rest of the world to be ahead of you at this, this is my opinion. I have this sense of urgency to always try something new and experiment and share my findings. [01:01:15] Dan Smigrod: It's very exciting because I want to invite you both back as soon as you have it and you've had a chance to play with it and talk specifically about, I'm going to ask you the question about Matterport + Apple Vision Pro and what's the vision for making money, saving money and saving time. Emily, Before we say bye, your final thoughts on perhaps Matterport + Apple Vision Pro? [01:01:42] Emily Olman: I think that my final thoughts are: I've been in this space since 2015 in VR, but my senior thesis was on cyborgs and the human-machine relationship and how computers help us and how they might hurt us. I think we've never been in a more exciting time to be seeing a complete renaissance of media and a renaissance of the way that we think about technology. For me, Apple completed this with Safari, unlike with everything being connected and us being able to have WebXR on any device easily in any browser. It completes something in this next generation, and I think that this is that moment. I'm excited about it and I've been a longtime fan of XR experiences and I'm just excited to see what I can make with it. [01:02:50] Dan Smigrod: Awesome, and Paolo your final thoughts on Matterport + Apple Vision Pro? [01:02:56] Paolo Tosolini: I think like Emily said, you just said the word I was going to use, the Renaissance after this VR spike, hyper/winter, I think the technology is there, the opportunities are there. We have a large user base of Matterport photographers, so they are eager to try something new to differentiate themselves. The theme, why not jump into this now that there is this possibility? It's expensive, so I understand that people will want to wait a little bit to see what's going on there. But for people who really believe in spatial computing, I think buying it on day one makes total sense, so that's what I'm going to do. [01:03:40] Dan Smigrod: I can't wait for the future to arrive! Paolo, Emily, thanks for being on the show today. [01:03:47] Paolo Tosolini: Thanks, Dan. [01:03:48] Emily Olman: Thanks, Paolo. [01:03:49] Dan Smigrod: We've been visiting with Emily Olman, CEO and Chief Media Officer of Greater San Francisco Bay Area-based Hopscotch Interactive (www.HopscotchInteractive.com) and We Get Around Network Forum Member name: @Hopscotch, and Paolo Tosolini, Founder and Creative Technologist for Tosolini Productions in Bellevue, Washington. (www.Tosolini.com) and in the We Get Around Network Forum member name: @Tosolini [01:04:19] Dan Smigrod: For Paolo and Emily, I'm Dan Smigrod, Founder of the We Get Around Network Forum, and you've been watching WGAN-TV Live at 5.
	1 year ago	Post 2 • IP flag post

WGAN Training U: Matterport + Apple Vision Pro: Imagining the Possibilities18832