WGAN-TV: WGAN-TV Live at 5: The Art of Matterport and XR Mash-Ups by Tosolini Productions with Bellevue, Washington-based Tosolini Productions Founder and Director of Innovation, Paolo Tosolini (@Tosolini) | Thursday, 6 May 2021

Video: Example of Tosolini Productions mashup. This video explores a #nocode​ solution to create dynamic pop-ups for immersive 360 virtual tours. When a tour gets populated with many hotspots that link to product information that change often, updates can become challenging and time consuming. Thus the need to templatize the pop-ups and leverage an external database to separate the presentation layer from the data. This video includes a mashup of Matterport, Matterport overlay, MPEmbed, and Bubble for the database.


Transcript: 101-WGAN-TV Art of Matterport & XR Mash-Ups by Tosolini Productions

Hi All,

Transcript below (Video above)

How might you mashup Matterport with other technology?

On Thursday, 6 May 2021, Bellevue, Washington-based Tosolini Productions Founder and Director of Innovation, Paolo Tosolini (@Tosolini), did a show and tell:

✓ 101-WGAN-TV The Art of Matterport and XR Mash-Ups by Tosolini Productions

Paolo shared how some of his mashups were made, including the technology and process. Many of Tosolini Productions mashups include Matterport. Some mashup other 3D-360-VR-AR-XR technologies.

Tosolini Productions has created 50+ virtual tour mashups since 2015: many of which have been posted to the We Get Around Network Forum.

Here is the Tosolini Productions PowerPoint Deck - with YouTube links - to the Matterport and XR Mash-Ups Paolo shared with us.

Paolo previously presented Tosolini Matterport mashups on WGAN-TV Live at 5 three years ago (February 2021). A lot of new Matterport integrations have been created by Tosolini Productions since then.

About Tosolini Productions (from their website)

We are an award winning creative studio with a passion for business & storytelling. We proudly craft interactive digital experiences that help clients connect with their audiences. Built on a spirit of innovation and research, we leverage the latest technologies to deliver across a variety of platforms

Best,

Dan

Tosolini Productions Links

✓ Tosolini Productions website
✓ Paolo Tosolini on LinkedIn
✓ Paolo Tosolini on Facebook
✓ Paolo Tosolini on Twitter
✓ Tosolini Productions YouTube Channel

Best,

Dan


Virtual Tour Mashups by Tosolini Productions

--

Transcript (video above)

- Hi all. I'm Dan Smigrod, Founder of the We Get Around Network Forum WGANForum.com Today is Thursday, May 6, 2021. And you're watching WGAN-TV Live at 5. We have an awesome show for you today, The Art of Matterport and XR Mash-Ups by Tosolini Productions and here to visit with us on that topic, Paolo Tosolini. Hey Paolo. Good to see you.

- Hey Dan. Thank you for having me here.

- Awesome.

Thanks for coming back on the show. Paolo is the Founder and Director of Innovation for Tosolini Productions based in Bellevue, Washington. If you're a regular in the We Get Around Network Forum you know that over the last probably five or six years Paolo has been posting his innovative mash-ups.

He's done more than 50 plus mash-ups of Matterport and related XR, AR, VR whatever you want to call all these new media. And it's so exciting. I invited him to be on the show to show us some examples. So, Hey Paolo, before you jump into showing us The Art of Matterport and XR Mash-Ups by Tosolini Productions, could you first tell us about Tosolini Productions?

- Of course. And thank you Dan, first of all let me actually use a little bit of a PowerPoint just to introduce the topic and also what we do. So my name Paolo Tosolini, I'm the Founder of a digital agency we call ourselves, we like to call ourselves also an innovation lab.

And we focus on business storytelling using emerging technologies and which for us means design and development of touchscreen solutions for events and trade shows and lobbies, 3D virtual tours. And we have, Matterport and other technologies and virtual reality VR and augmented reality AR.

For us, like I said, it's very important ... Research is one of our competitive advantages. And that's what we're going to talk about today. We are not traditional real estate photographers but we use the same tools and thanks actually to you Dan, and your Forum at the beginning is where I learned first about Matterport that you convinced me to buy into the ecosystem and then to share back to the Community everything that we were learning along the way.

So our clients are slightly different than the traditional, real estate photographers. So we work with corporations, Microsoft, Accenture, Deloitte. I come also from Microsoft myself from having worked there for a few years. But enough about us. I just want to talk about the business of being an agency and then maybe also comparing it to having being a product startup.

So we are in the services business. It means that we sell -- we promote our creativity, our development design services and being in a services business, as you know, Dan and probably everybody else who is in a similar business has pros and cons. It's not as scalable because you cannot clone people.

You have great employees and then how do you clone them? It's not possible yet. And, but the good thing is that every day is a new day. It's a different project. ... Two days are never, never the same. We sometimes work on 3D scanning the other day on touch screens.

Some other day we were traveling the world doing something else. So to me, it's exciting, the variety of work. Which brings me to the topic of this live stream, which is how do we market ourselves and what is like being in the services business and tell the world we can do the things, we can innovate that we are all about using the latest technologies.

But, trust us that we can do this, this project. The way we found effective is using prototypes. And what is a prototype? Prototype is a proof of concept that aligns usually with some business needs of a particular client, or simply it's a proof of using a particular technology that we have never used before. But there is something about creating a prototype so that could be more or less effective.

And the way that we found more effective is not just taking a technology that we didn't know before or we just want to know more and just do something with it. It's taking a technology and creating those combinations with another technology possibly or maybe other two different technologies in a novel way.

This is what we define as mash-ups, is all this combination of different technologies, put them together in a way that probably haven’t been done before. Not only that, we also like to align two business use cases or a particular brand ... when we can or time the release of the prototype ... with some sort of bigger event. We benefit from the tide that event is creating.

So with that said, this is our mash-up strategy. And I have a few of ... ... I think 10. We have done 50, right Dan? You mentioned that, but then I just took ... I just took the 10 relevant mash-ups that I think you might enjoy.

- I happen to enjoy all of them Paolo and I do watch all of the ones that you post in the We Get Around Network Forum. And I often watch them more than once because I'm just always... I find them not only just awesome, but I find them inspiring. It's like, "Wow!" The way you've figured out how to mash-up these different platforms in an unexpected way to create value for the viewer beyond what each individual platform might contribute to the viewer. But when they're mashed up then it's kind of like one plus one equals a hundred.

So I am glad you're giving us the highlights; but actually your 50 plus are amazing.

- Oh. Thank you, Dan. I know you're a big fan of these mash-ups. And by the way, I want to give credit to my team. So while I may come up with some of the ideas, my team is really doing a lot of the execution. And so it's a shared credit here with the rest of the team.

So I'm going to start with this mash-up. It's an augmented reality mash-up and you will see a lot of these demos are going to be using Matterport data. And this data, usually it comes in the form of a 3D OBJ file that you purchase with a MatterPak.

And this is one of the early things that we have done is -- ... What can we do beyond just the virtual tour? What can we do with this MatterPak; with this OBJ file? Can we plug it in somewhere else in another app? And not just plug it in, how can we combine it in different ways?" So here is an example of an AR app that we built using Spark.

So Spark AR is an authoring tool by Facebook, that allows you - it's a free authoring tool - that allows you to create all of these filters for face AR. For example, adding big glasses or whatever, as you're looking at the, for example, through Instagram. So we took that authoring tool, we pulled out some data from Matterport, and we thought about the use case about accessibility. Accessibility is a big thing for corporations.

I mean in general, it's the right thing to do. Thinking about all the people with disabilities who cannot fully take advantage of a particular interaction with a computer.

So we decided to create this mash-up. I'm going to click on the link here on every page and I'm going to go full screen with my video here. So imagine on your phone taking a look at the Matterport OBJ file and being able just using expressions of your face to change it and to interact with the model.

In this case, for example, let me go back and let me replay the beginning. So imagine if you can recognize your face-

- So, Paolo, just to be clear I don't see anything moving on your screen. Should I?

- Oh, okay. So let me see. Ooh, my mistake. I didn't share the whole screen. Did you see the slides before? Sorry.

- Yeah. Now we're seeing.

- Okay. Were the slides visible before or not?

- No, not at all.

- No.

- So take us to the beginning of this. I think because we were prepping some technology. My mistake because I asked you to prep yet some additional technology before we got on the show today. And I think I threw you off on your setting. So let's begin at the beginning on this slide here.

- Okay. So this is the slide. Okay. This is the slide relevant to this particular demo but anyway, thank you for reminding me that I wasn't sharing my screen, my fault, this case. And then ... Here is the actual demo. So this is running on a phone. ... It's using Spark AR.

And imagine if by using facial recognition you can control the 3D model that is running on your phone. Okay. So you can blink your eyes and you can change, for example, a particular feature of the 3D model, or you can rotate the model just by looking left and right, or through the smile you can change the color of the model.

So this is one way that we could help on the accessibility side.

- That's amazing. Did you have anyone on your team -- or be able to show this to someone with a disability that could benefit from this?

- It's a good point. We haven't tested it ... in real life. This is something that we could probably do. We were so excited about the opportunity to just create this , but we haven't gone the full way of actually testing if this is, where we could further enhance the concept. It's a good idea to do it next time probably.

- Okay.

- Okay.

- So I love that because ... you make it look so easy and yet I'm sure there's a lot happening behind the scenes to actually achieve that result. Is there actually some place we can go to actually experience the AR yourself versus the video?

- So I... Let me see if we release this... We should take it offline. I wonder if, even if I put the link on my YouTube channel. We should take a look later, but this is a good question-

- Okay. If not, if you if, excuse me, if it is still available, if you could send it to me, we'll put it in the We Get Around Network Forum, WGANForum.com when we post the rerun of today's show, so that if it is live -- In fact, you're kind enough to offer to send us the deck.

We'll put that in the We Get Around Network Forum. And maybe if that link is available, you could either add it to the deck or just send it to me separately. We will post that.

- Sounds good. I'm happy to share anything that I can share, as you know, I'm sharing. Okay. I want to talk about the next mash-up. And this one is about -- still AR, augmented reality.

And if you're familiar with the merge cube -- So, the merge cube is a physical cube, is it's kind of a rubber cube, one of the stress relief cubes but the goal of the merge cube is really to have six faces that could be great trackers for augmented reality.

They do have an SDK that you can build upon the cube but it's meant for education. If you give it to a kid, the kid takes the cube, they run an application on their phone and they see through their phone different things on every face of the cube. It could be a game. It could be whatever.

What we have done is taken the cube and made it into a business card; an AR business card using a technology called Vuforia which is a software that allows you to track images for example, for AR and Unity which is a game engine. It's a 3D real-time game engine.

- I'm guessing that Vuforia lets you set each of those six sides as targets so that it recognizes the target and then goes off to a database in Unity to play it.

- That is correct. And in fact ... Let me start from the beginning. Here is me just showing; setting the context. Here's the cube and everything that you're going to see from now on is seen through the lens of the phone. So our VR designer put together this experience and here it is.

Every face talks about one of our business [services]. for example, 3D scanning, touch screens. It's more like a promo of what we do through AR at the same time.

- So this may not be obvious to our audience. So I just want to kind of re-emphasize this. This is holding up a smartphone that is running an augmented reality app, pointed at the cube, and now you're recording video from the phone?

- Correct. That's correct. This is what you could see from your phone, but in reality you're holding the cube. So everything is being processed in real time. And by the way, the background there is Seattle. So we are really outside/outdoors and here it is, blending reality to augmented reality.

- Again, that's awesome. And is that a link that we can experience with an augmented reality app?

- Not for this one. This one was on the phone of our designer. For this one, you have to just enjoy the video. Okay.

- Cool.

- But that is a great example of mashing-up reality - augmented reality - using these tools Unity, Vuforia and this merge cube in an unexpected way. And in this case it turned into a promotional piece for Tosolini Productions. Awesome. ...

This was because we were curious about the merge cube, but that you could accomplish the same result with no cube. You don't need the piece of hardware or the cube itself, you can have AR with paper and so on.

Okay. So this one, this is about collaboration. And we wanted to experiment how to mash-up Matterport OBJ files with a collaboration tool like Mozilla Hubs. Mozilla Hubs is this web-based space where you can have avatars.

I think you can have 15 or 20 people maximum in this virtual chat room. And you can view it through your browser through ... let's say web VR at this point.

But we wanted to take it to the next level, which is ... what happens if we create a space, we customize our space using an OBJ file from Matterport. Not only that, what happens if we put inside the OBJ file another OBJ file another Matterport plus a 360 photo.

So let's see, let's see what that is.

- I can't even imagine what that is. So I'm glad you have a video to show us.

- Yeah, exactly. So sometimes, yes. Oh, look at this. ... Are you still there?

- I'm still here and-

- So PowerPoint restarted but I'm going to-

- Okay. Now we can see a video playing.

- Okay. So, okay, perfect. So I will have to restart my PowerPoint later

- Okay.

- because it crashed.

- Yeah. I know we have so much technology that we use to bring our WGAN-TV Live at 5. viewers all this innovation, and I'm sure it's causing some challenges for you on your end, so thank you for-

- Sure. And it's all about testing things and breaking things. Okay. So let me... Let's take a look here. So what do we see here? There is an environment.

This is a 3D scanned home. Okay. And then we added some objects, actually our XR Specialist, Maria Sanchez who cannot be here, did this for us. And then inside the 3D space, what do you see here? It's another model.

It's a kind of an inception, I don't know what you want to call it but here it is, Matterport within Matterport. And now we're putting our head inside and just looking around and then you're seeing in the kitchen over there, so here is an avatar. So we were together.

Now you see a 360 and then we need to get inside the 360. Right. Because that's what we do. And suddenly you see a photographic view of the kitchen in there. And again, this is using all tools that are free. And in Mozilla Hubs, they give you an opportunity to do the authoring and in the OBJ files and it's all about experimentation.

- On this one, were you thinking of any business use cases or this was more, "How can we mash-up all this stuff and just show that it's technically possible?"

- Well, we didn't think about, let me close these and let me reopen the PowerPoint in the meantime, as I'm doing this.

We didn't think about a business use case yet. It's a good question, but there could be, there could be some.

There could be some for education. So for example, when we are talking to the Museum of Flight in Seattle.... The Museum of Flight -- they are closed because of COVID -- they're thinking their education department might benefit from having this kind of space where you could import a 3D model of an airplane and have kids joining you and have the docent explain inside this chat room what that particular airplane, the history of that plane.

And so I think there are business cases. Maybe not directly for us.

Maybe we could help somebody create those spaces for them. You will see a lot of that then. You will see a lot of our mash-ups. Sometimes we don't start necessarily with an ultimate monetization goal. They start out of curiosity sometimes: Or, desire to enter into a space that we have not entered before. It's all uncharted territory. So I want to-

- Yes. Forgive me. Because you know me so well that I'm always thinking, "Okay, well how can I make money with that? How can I help clients make money with that?"

- And-

- I will live in the moment of curiosity and innovation without asking that question; how can I make money with it?

- But I appreciate when you ask me the question because it brings me to reality, why do we spend this time to do this experiment? And sometimes it's just pure curiosity of just desire to share with the Community, something that hasn't been done. So-

- Which is awesome because it certainly says for any company, certainly on the scale of Accenture, Microsoft, "Well, we're interested in doing the following but we haven't done it before. And you haven't done it before. Why do we think that you might even be able to do that?"

- Yep.

- Well, that's what we do. We do things we haven't done before and we figure it out.

- Yeah. And that you have to be... you have to have a lot of tolerance for failure when you discover... When you do these kinds of prototypes, right.

We don't know what we don't know. But if we wait for somebody else to take the first step, it's already late and then you're not an innovator, you're a follower. But we like to feel like: to be the first sometimes to enter into a space, or by doing these mash-ups. It's just a personal satisfaction, I guess.

- Yeah. And I love it. And I love that you're willing to risk failure in order to develop something that no one else has done before and it's truly innovative. And I will hold off asking you my questions. "How can I make money with that?" "How can I help my clients make money with that?" "How can we help the Forum Community make money with that?" ... I'll live at the intersection of curiosity, innovation, mashed-up with technologies we've heard of, and those that we haven't heard of that you're introducing us to.

- Good. Good. Good.

But please keep reminding me how to make money. This is important, but okay.

Another mash-up. This one ... is pure curiosity and we stretch it to align it to a goal of a corporate goal, for one of our clients, Microsoft. It was completely unsolicited.

We wanted to surprise their team that thinks about environmental responsibility, corporate responsibility and because we are all about storytelling using emerging technologies, we said that, "How can you tell the story of a company that is committed to be green and then very responsible with the environment?" Using WebXR.

A WebXR is the set of technologies that allow you to create immersive environments that without any download, it's just in the browser. And what you're going to see here is that not the super high quality 3D graphic, because again we are in a browser, there is not...

You have to be light with the weight of the whole experience and so this is a little bit more blocky right there, but the goal is that you can explore this place. It's a little town with features that remind you that there is the windmill there, there are links to a website for Microsoft sustainability, you click a space.

And again, this was a technology experiment that aligned it with a story from a corporation. Now we used a framework called A-Frame to put the whole thing together. And it's, and again, this place in the browser, just in a simple browser.

- Yes which is amazing. We're not downloading an app. We're not

- Nothing. You download nothing . It's just plays. Again-

- It's interactive. And it's within the browser.

- It is interactive. Yes. And this is where, in my opinion, things are going. It's more and more in the browser. You don't want to all the time, download an app it's taxing to download an app.

It requires an effort from the person. Again, the best experience, of course, it's always with an app because it leverages the whole full power of the device. However, sometimes you just cannot do that. And we were exploring what can be done with story business, storytelling with WebXR.

- Awesome.

- I know you like traveling Dan, and this is a travel-

- I forgot what that's like, I think I forgot what that's like. What is that like?

- And then you have been there probably in itself. What is this? This is another demo. It's a mash-up of a space built in unity, Unity 3D.

Again, this is a game engine and professional 360 photos done by a friend of ours that we all met at one of the, one of these meetings right? IVRPA, what it's called, meetings that we've done, Pietro Madaschi, so here's an Italian photographer, 360 photographer.

And I talked to him and I said, "Pietro, I have an idea. I don't know if he was going to work but I need the beautiful photos which I don't know how to do but you are on the spot and you have them. "Can you please send me the photo?"

So he sent me the photos. And what we built is a portal, a portal to Italy that you can experience through your phone just by walking in different directions of your room. And let me show you what it looks like.

- This is probably one of my favorites of the 50 plus apps that you've mash-ups that you've done.

- Yeah. So imagine you select the three destinations and you're in the lobby, and now in front of you there is a portal, there three portals, right? All augmented reality. And you physically walk towards these portals and now you are inside. You are in Italy. Now you are in Venice.

And as you look around that, you're looking at the 360 photo. Again, everything you're seeing is through the lens of your phone. And now you're in Rome. And this is like virtual travel. Maybe during the pandemic.

- And there's, with the pandemic this may be the only way that I'll get to revisit Rome and Florence and Venice.

- Well, I hope that you, you get to visit live soon, right?

- Yes. With short of visiting in person this is really quite an amazing augmented reality experience of feeling like we really are there. Did you happen to take this one and then also do a headset for virtual reality?

- So all the photos are not mine and are from Pietro. I haven't... So this is actually done in Unity for a phone. And so you really need to walk closer to the... Physically walk in different directions to enter the portals.

- Hmm.

- You could teleport yourself. We haven't done that, but technically you could teleport yourself into the different portals with the headset, it's just that it was out of scope, but we wanted to do-

- Yes.

- a mobile experience.

- Perhaps when Apple comes out with their augmented reality glasses there'll be yet another dimension to add to this AR travel portal.

- For sure.

- For sure. I look forward to that time. Okay. Back to Real Estate, which I think our audience is more interested in. "Okay. "Paolo you show me Italy; show me all this kind of stuff. But ... show me something about real estate." Okay.

This is for you then. This is a mash-up that we have done using an app; an app that allows you to alter augmented reality on your phone. It was, I think, one of the first ones that... It was called Torch AR and I'm talking about the past because unfortunately they had to close shop.

So Torch AR what you're seeing here is almost everything has been altered on the phone which to me is amazing other than the SketchFab part which I had to do it on a computer.

And it's some Blender things but putting together this experience was done on a phone. And I love when authoring happens on the device. So what you're going to see is an AR listing, hypothetical listing. And this is by the way, my backyard you're seeing in the background there.

And everything happens through the lens of my iPhone. And you're going to interact with elements that are in AR through the iPhone. So let's take a look. So imagine that you're walking, and then you see this ad in AR. This is a video that has been embedded into the AR experience for Matterport. And now you say, "Okay, I love this home but let me see the floor plan." I click the floor plan. It turns down horizontally, there are two photo spheres and you click on one photo sphere and suddenly you are in the kitchen.

- Wow!

- And it says, "Okay, I want to see the dining room." And then you click the second photo sphere. And now you are in the dining room.

- Wow. That was pretty seamless too.

- Yes. What I have to say, Torch AR had, it was very powerful, but it took a little bit of work to make it smooth and seamless, but it was, to me it was an amazing piece of technology that it was probably ahead of its time like many of these technologies, right? But again, a mash-up of Matterport, floor plans, videos, AR everything on the phone. That was our goal.

- Every time I look at one of your videos, what comes to mind... When anytime I look at one of your mash-ups what always comes to mind is it must be incredibly hard work to make it look so easy.

- I have say we don't disclose how much time, how many weekends and nights we spend there, because for us is especially for me, it's... I love doing the things. It's almost like if innovation it makes me, it motivates me.

So to me, it's my hobby and my work coming together. In fact, my wife sometimes complaints that, this is too much of the same thing, but hey, that's how it is. Okay. AR shopping, this can be monetized then. So I'm, getting closer to your line of thinking here. Imagine the opportunity to walk into a virtual tour and experiment some items.

So in this case we 3D scan and eyewear shop and play with the glasses in 3D and then potentially purchasing them or even try them on yourself. The mash-up here is about Matterport; a traditional virtual tour. E-commerce it using Spark AR to create the effect of wearing the glasses and then another third-party platform called Genial.ly to create the more customized MatterTags So let's take a look. This is the video.

This was done by our XR Specialist, Maria. So you will see her in the video. So we're are walking and we spot a nice pair of glasses there. Let's learn more. You click on it and this load SketchFab. So this is the glasses loaded in SketchFab.

So you can manipulate them and rotate them where you want. And now there is the e-commerce part then through Genial.ly which is one of the third-party approved vendors that allow you to create a rich media in MatterTags.

And then the final part is launching the filter in Instagram. Let me go back here for a second, launching the filter the Instagram filter that gets to try, allow you to try those glasses on your face. So Spark AR is the authoring tool free by Facebook that allows you to do this kind of thing. So all coming together e-commerce right. So I can see monetization opportunities here. What do you think Dan?

- Yes, absolutely. I walked into the store. I found a pair of glasses that I thought I was interested in. I tried them on. I looked at myself in the mirror. I went, "That looks great." I can now go buy it. And I've never left my house.

- Exactly. And you pay.

- That's awesome.

And you would say, "Well, who would want that?" Well, there's a lot of us today, who would not go into the store as a result of the pandemic. So, and I suspect that these kinds of innovations you're doing, I sense is the amount of both innovation and adoption of innovation that has occurred in the last 12 months might have taken another 12 years had it not been for the pandemic.

So I think these kinds of new ways of new media as you describe business storytelling with emerging media just seems like the natural progression of where we should be in our everyday lives anyway.

- Mm hmm.

- And more and more of us are comfortable with making buying decisions without ever leaving our house. I know we're even ordering, from the groceries we order, from and the furniture we've ordered, this is the next progression. "Okay, well, I can just try these glasses on in AR why not?" And-

- Right.

- For the average person, they don't need to know that you've mashed up 10 different technologies to accomplish that they just know from a user experience, "Yeah. That's a great user experience." So-

- And I'm sure in the future all those 10 technologies may become maybe two because who knows, maybe Matterport and another third-party will blend three or four of these technologies together, but at that time I had to use so many different technologies.

I have another example related to real estate here, Dan. This time... So this is actually a couple of years old. And if you remember Magic Leap that awesome headset that was going to become a consumer headset, which didn't. So we bought one of those because we had HoloLens that we wanted to see the difference there.

And then we decided to create another Real Estate demo mash-up, And it was the time when Google Earth Studio also was released. And we said that we cannot leave Google Earth Studio out of our mash-ups. So let's put everything together.

So what you're going to see is the experience through the glasses of Magic Leap. Imagine Magic Leap is this device that you can see through. So you see the reality but you also see augmented experience on top. And there is a Matterport model...

- Oh everything is done in Unity, by the way. Actually, you know what? Let me show you, because I think it's best to be, to guide you through. Okay. So this is what you're seeing. Look, here's a building that forms in front of your eyes.

You pull like a drawer, one of those floors, and then inside that you see people, there is a Matterport model. That's the Matterport model with people walking around. This is how we did it in Unity. And this is what you're seeing through the Magic Leap.

And you control it through a special command, like a handler. So our XR designer, Michael Gelon did this and it took him some time because he had to learn about the Magic Leap; finding their SDK. So, it takes some time to build this.

- I also imagine on Google Earth Studio... I want to say we were both in London for Google Street View Summit where Google Earth Studio premiered. And I'm sure it was in beta while you were probably working on this. And so everything about Google Earth Studio it probably didn't work the way it was supposed to during the early days. So I guess ... kudos

- Good point.

- to you and your team from not only innovating with new media, but innovating with new media, which really hadn't been announced at the time that you were actually working on this.

- And yeah, so we always try to jump on a new technology and just leverage the fact that it's so novel. And that's one of the factors about our mash-up strategy. Is there something new? Let's jump on it to see if we can make a good demo but not just with one technology let's add two or three of those.

- Can you go to the, to a screen that has Google Earth Studio, just... I think you had a map, an aerial view of a location.

- Yes. So this is the building all formed in Unity. You pull, this is Matterport OBJ file in Matterport augmented with some people walking around and this is the part of the-

- Right there. So for our Matterport and related Community in the We Get Around Network Forum; yet another platform for how would you describe Google Earth Studio?

- Imagine the ability to create videos from Google Earth. You can do a fly through from the above and then zoom in on a particular area. It's more like an animation tool for Google Earth. You can have a camera. Make the camera rotate and have a lot of visual effects-

- And then annotate within that environment. So you have a location and now start bringing in your assets.

- That's right. And on your [www.WGANForum.com], I've seen a lot of Members that are using it to tell a story about the property in a video, right, Dan? So the video starts with Google Earth Studio. You zoom in and then at the end, you go into the Matterport. Great blending of technologies.

- Mm hmm. Cool. Thank you.

- Cool. So let's talk about storytelling in 360. So this is something that we noticed an increase in interest during the pandemic. You could do a virtual tour using real photos which is the most common use or you can create synthetic environments.

It could be, for example, a museum that is drawn in 3D that is modeled in 3D or it could be a story about the business.

So what I'm going to show you is a product presentation that a company here in Seattle called Sensoria, asks us to do for them ... to present a new product to various clients. This has been done in Unity. Unity is again, is this 3D engine.

We extracted the 360 from Unity imported into a 360 authoring tool. In this case, we use SeekBeak which is a by the way, great platform, in my opinion. And then we added hotspots. So let's take a look at the first demo and then I'm going to show another one.

So again, Sensoria is a startup. It's an IoT startup. They build sensors that you can put into clothes, shoes, socks, and so on And they are taking these sensors to help disabled people, to monitor the amount of exercise that they do while they are on a wheelchair.

Those sensors talk to the cloud and they transmit information to the doctor. And the doctor knows if the person; the patient has done their exercises or not. How do you present such a complex system? So we created this, this space.

It's a 360 menu where you look around and you decide if you want to go left or right. Enter into the doctor office or go to the patients home. And it looks like that the people like the idea of using this kind of storytelling.

On that note, let me show you something else. Same principle. Can we do a virtual expos using this kind of technology, synthetic 360s? Here's another proof of concept we have created. Imagine you have a trade show and today they're not happening. Right? But we built one in Unity.

All of these are modeled in 3D with hotspots that you can click. Maybe I can also, after the video I'm going to show you the actual experience. So then if you have questions I can answer those. But the idea is to have different booths that represent different companies, those booths feature hotspots and navigation, using a common language.

The common language you will say is this orange border. Every time you see this orange border it means something is clickable. And actually, let me go to the experience so you can take a look. This has been done in Unity, but then has been assembled using SeekBeak.

So this is a 360 and there are different ways you can navigate. You can navigate sequentially, or you can go to a map. So let's go to a map. Let's pick, I don't know Microsoft. These are all fictitious companies right there. And now I am in the booth. The booth of this company is called Microsoft. But if you click any of the videos, maybe it's Microsoft. That's okay.

There is some alignment there and you can essentially brand the space. You can go wild with some ideas, for example, let's enter in VR. So this is an Oculus, right? The headset. Let me click here. Let's see what happens if I click inside. And now I am simulating being in an air balloon. And again, this could be some sort of story about travel and you have infographics that pop up.

- But what I love about this one Paolo, and particularly because of the pandemic, which has meant no trade shows. So for all the people who are comfortable with going to trade shows you've now extended the metaphor into a virtual trade show. So anyone that's been to a trade show feels as comfortable because the experience is identical. You've made a digital twin of a trade show on a desktop.

- That's right. So here's a pizza for you, by the way.

- Thank you. Extra cheese, please.

- Now, to get to this point. Let me tell you the steps that we took, because this is the third iteration. We started with a simple booth. It was simple. Imagine a booth that you can just navigate.

This was our first attempt to have a virtual expo. And this can work, but it has a problem. And the problem is it looks empty, right? And you say, "Ah, okay, I understand the concept but let's go to the next level." The next level we took is experimenting with multiple booths at the same time.

And then again, all done in Unity and SeekBeak. We represent here some companies, this our booth, but again, we looked at it and we said that it's missing something again. It's missing the action. The people.

Everything that we are used to seeing when we go to a trade show and this is when we thought to add the people having fun. Eating. Everything that you could find in a trade show. And now I think we are getting closer to what an experience might look like. It will never be the same as being at a trade show. However, it's getting in that direction.

Oh, I have another one. This is actually my last one.

- I don't want to just move off that without just saying, I love that. That's amazing. Once again, you've taken something that it's probably incredibly hard to accomplish and make it so simple. So again, from the user experience, it's just, "Well, yeah I'm at the trade show. I'm walking around and things I would normally be able to do at a trade show, I can still do."

- I agree. And that prototype took us a few weeks of work but the things that we discovered by doing these prototypes is that the user interface needs to be consistent as you navigate. How do people realize what is clickable and not? And so this is once we create something that works, it means that to get there, we probably did another five steps before and failed all along. And so, yes-

- I bet, I think one of the great things about seeing a virtual trade show is there are so many companies that spend so much money on a booth in design, and anyone who's ever been involved in any trade show knows it's a bazillion dollars to participate in a trade show.

I would imagine, hopefully they spend a bazillion dollars with Tosolini Productions, but I'm going to guess it actually wouldn't cost as much as it would to go have the $1 million or $2 million booth physically at a real trade show that they could have their own virtual show all their different divisions for a fraction of the cost and have it take place virtually. And for all those people that can't travel internationally can still participate in the show.


So I just look at this and go, "Gosh, I hope there are a lot of companies that when they look at what you do go, "Wow!" Our problem is we can't do trade shows or we can't do in-building touchscreens. Everybody's at home. So what do we do? Well, you have all these different solutions that you've been pioneering for quite some time. I hope that ends up translating to companies that are interested in Tosolini Productions and coming to talk to you.

- Yeah. Then, let's say that again, not all the prototypes. I would be amazed if every product leads to a project that we have a very high rate of failure for these prototypes. but the failure is not as much about the concept but the timing.

We are getting interested in this 360 storytelling, that we did three years ago and only now is frustrating three years ago to feel like, "Oh my gosh, why don't people get it? Why don't these companies get it?" And now we are getting those requests.

So there's a timing issue of sometimes being an innovator that you have to live with them, but that's fine. We enjoy this. Let me talk about, speaking about timing. This is good timing, because in this example it was all about timing.

This is an AR demo that we did for a book author. His name is Pascal Burnett. He's an influencer on AI, artificial intelligence on LinkedIn. I bought his book and I have been following him. And he liked some of our previous work. He came out with a book, a book that he has been advertising.

He has a lot of followers, and we thought, "Can we do something for him?" And time the release of this demo is completely unsolicited. He didn't know that we did it for him and just sent it to him and said, "Look, we have done this demo for you. If you like it, talk about it. If you don't, don't worry." Right?

We used Adobe Aero, which is the AR solution to author AR with, by Adobe: It's still in beta. And there was a combination of Illustrator and the topic is supporting a book launch. And the cool thing is that he liked the demo and he retweeted on LinkedIn.

And we got 25,000 views of this, this video. And again, this is done in Adobe Aero. So the infographic there is clickable, it's 3D. Vision. And you click on any of this topic and it just pops up. And then once you, you think like "Oh my gosh, I love the concept.

Let's learn more about the book." You click on the little robot, you get the book and then you click on the book and you go on Amazon and you can buy the book on Amazon. So that was his simple demo. But again, something that it was about timing, the prototype to another event and it worked. It's Guerilla marketing. I call it.

- Did you use your Amazon affiliate link to monetize?

- Dan, I need to learn from you how to really monetize these things. No, I didn't. I didn't because I didn't think about it. But that was my last demo. But I have something else. I don't know if you feel that we can-

- Yeah. Could you go back to the, your last slide there while you're setting up?

- Oh, yes. Here it is Dan.

- Thanks. Thank you. So while Paolo is going to do a live demo for us on an interesting mash-up, but if you're interested in getting in touch with Paolo - Tosolini Productions website is Tosolini.com Tosolini.com In the We Get Around Network Forum @Tosolini and his email address is: Paolo@Tosolini.com And Paolo is going to send us his deck which has all these links to YouTube videos. In fact, why don't you go ahead. I'm going to take people to our QR code for something. Should I do that now? Or you want-

- Yeah, you can do it now, if you want that. I'm ready. Whenever you want. I'm ready to do another.

- Okay let me, I'm going to take over to share the screen just for a second here. I'm going to go to the We Get Around Network Forum WGANForum.com then I'm going to scroll down.

This is the Forum post we were promoting today's show. What I wanted to show here was that Paolo was put together and an app. What'd you call this a desktop app?

- It's a web app, actually.

- A web app.

- It's a web app.

- So a web app doesn't require a download. You don't need to do the iOS store. The Google play store. This web app has 50 plus virtual tour nash-ups by Tosolini Productions. And if you go, you can either scan this QR code and it will launch this on your smartphone.

Or you can come to the We Get Around Network Forum, WGANForum.com ... go to the search box just type in, let's say: Art Matterport XR or search: Tosolini productions You'll pop up this WGAN Forum post.

And then there's a link right down here to go to this web app. So it really is awesome. So if you love what you're seeing, you can go, literally ... this will make it easy to see 50 plus implementations ... mash-ups that really are the definition of business storytelling with emerging media.

Paolo, are you going to do a live demo for us? You want to tee up what you're going to do?

- Yes. So what I'm going to do. So as you know, Apple is getting into putting the LiDAR sensor into the new iPhones and this LiDAR sensor are depth sensors that allow you to capture a space in 3D.

And there are new apps coming up that are really doing a great job in capturing in a matter of minutes a space.

And again, this is the beginning of something bigger in my opinion, there's no competition yet with Matterport necessarily and so on, but you can tell how important it is having this kind of sensors on your phone. So what I'm going to do is mirror my phone onto Zoom using a little app called, reflector.

- I don't know what Zoom is. ;-) This is WGAN-TV Live at 5!

- All right.

- You're going to mirror your phone to WGAN-TV Live at 5.

- Oh Okay. You're right. ... Now we are live on your show. Can you see it?

- Yes.

- Okay. I'm going to launch an app. So these are all apps that I like to play with that there is one that is called Polycam. So Polycam is a 3D scanner, essentially.

And as you can tell, as I'm looking around, the app is able... By the way, here is me. It's able to detect it and create some sort of mesh and create this polygon to record as a 3D model. So I'm going to press the record button and I'm going to attempt a 3D scan of my office.

And let's see here, what it will look like. The blue area that you're seeing --- It means that it hasn't been scanned yet. And everything else that is -- as I scan it, the texture shows up. Okay. So I'm pressing the stop button and this is the end result.

This is a model that doesn't have any texture yet. And I'm going to click the process button. And it shows in this beautiful animation all the space that the, the angles of the camera, and here it is, this is my office. Of course it scans just from my point of view.

It's not perfect by any means. However, this could be a nice doll house, if you're doing a 3D virtual, a virtual tour and you don't have Matterport, for example, right. Again, just another example of how technology is progressing so quickly that you can mash-up all of these things. By the way, I have a tutorial on how to add your own doll house.

Let's suppose you use 3D Vista as a... Or SeekBeak or you just don't have Matterport, for example, you could use your phone to create a doll house and then use that doll house as part of your virtual tour.

- That's awesome. Is that one of the items listed in your virtual tour mash-ups?

- It is.

- Okay. Good. So we can go,

- It is there. We can either scan that QR code or go to the website, go to the We Get Around Network Forum, click on the link to view your web app. And that'll be one of the items that's listed.

- That's right. And I found that creating these tutorials really helps a lot of people in the Community. So I'm having fun by doing these mash-ups

- Well, you're having fun but I think you, this Art of Matterport and XR Mash-Ups by Tosolini Productions, you're inspiring the Community. You're showing what's possible. It really is quite remarkable of what you've accomplished over many years.

I think from the time that I've known you I think it's about six years, the mash-ups that you've done have just been really remarkable, inspiring. And anyway, I just think of it as a gift that you've given the We Get Around Network Forum Community by sharing your mash-ups particularly the ones that are Matterport related or the ones with other 3D/360 platforms to show what is even possible.


And I think it's probably unanimous that everyone in our Community just loves what you're doing and what you're sharing. So I'm very appreciative for what you post in: www.WGANForum.com and what you've shared with us today. And you've been on our show in the past, Gee I think maybe three years ago talking about some

- Three years ago.

- of Matterport mash-ups then. So anyway, I know I appreciate it. This is actually

- Oh thank you Dan.

- Tosolini month on WGAN-TV Live at 5. Next week your wife, Francesca Tosolini will be on the show, "How To Do It Yourself: Virtual Staging of a Matterport Tour Using the CAPTUR3D Platform." Looking forward to that.

And I think this is really an example of taking a platform that's initially been designed to mash-up with Matterport, and most importantly take an online interior decorator - interior designer - to see how someone who is a professional designer actually uses technology in a way to accomplish a business use case for her clients.

So Tosolini month on WGAN TV Live at 5. How exciting is that? Paolo is there anything else before we sign off here? Is there anything else that you wanted to talk about? Something we haven't covered?

- No. First of all, I want to thank you, Dan, for giving me the opportunity for me and many other people to share their knowledge.

Your Forum I think is a great place for people to ask questions; but also to share ideas. And the more I found, the more I give the more I also get from people so it's like the whole universe is working in the right direction there.

And I would love input from... if you, if this audience has ideas about potential mash-ups that might be of interest to the Community I'm always interested in listening. I do have to share with you. So here is my list of mash-ups divided by priorities.

And so I'm going to add your ideas here and then prioritize but I still love to hear from you what would work for you.

- Awesome.

- Thanks.

- Paolo thanks for being on the show today.

- Thank you. And thank you everybody.

- We've been visiting with Paolo Tosolini, he is the Founder and Director of Innovation for Tosolini Productions based in Bellevue, Washington. He's done more than 50 plus innovative mash-ups, all that our business storytelling with emerging media.

You can learn more at: www.tosolini.com www.tosolini.com www.tosoline.com For Paolo in Bellevue, Washington. I'm Dan Smigrod, Founder of the We Get Around Network Forum in Atlanta, Georgia. Thanks for tuning in. I need a little thumbnail here for our show.

- Big smile. Thank you.

- All good. ;-)