Matterport Talkie Tour -- Feedback Wanted2943
UserName private msg quote post Address this user | ||
That hotter and colder thing sounds fun and useful. @Metroplex360 - You might test Google's OMNITONE experiment. They're trying to bring 3D surround sound to WebVR. When you visit this page, wait a few seconds and a 360 video will load. Various musicians will wander around a hall. As you rotate the video with your mouse, you'll be able to track everyone by their sounds. For instance, near the video's beginning, a singer on the right side of your hall may sing while a guitar player with loud footsteps walks in from the left. You'll hear both of them just as if you were there. As you rotate the hall, all sounds change appropriately, ie, if the musicians are behind you, you should be able to tell. You probably have to wear headphones to experience this and use Chrome also since that may have a higher probability of working. Here's an article about OMNITONE. If Matterport models could contain animations, like fireplaces and babbling outdoor fountains imagine the fun. |
||
Post 26 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
@Baxter I have no idea why Matterport haven't already released a Javascript API to allow us to write interesting extensions like the 'Talkie Tour'. Now as far as getting the location's audio to face or increase... that's quite complex. First, with the 'talkie', let's create a scenario where we had a Matterport Tour of a robot factory full of talking robots. When one runs away from a talking robot, I could indeed have the volume decrease with distance. I have the ability to create events based on the panorama, the pan, and the position within the model. In addition, events can be added when swapping view modes (if you note in my demo, there are voice events for dollhouse and floorplan modes). So in theory, the tour would capture the location within the mesh at the time the sound starts and augment volume based on one's position. In my published proof-of-concept, this wouldn't make much sense though @Baxter / @UserName - Where I'm a bit stuck on this idea -- especially on a 3D Surround Sound concept - is that a Matterport Tour is not a video 360. It would make sense if there's motion to have dynamic sound. Even simply having sound that trails off from the theoretical 'robots' in the robot factory example wouldn't be too useful in the real world -- more of a neat tech example, right? |
||
Post 27 IP flag post |
Olivius private msg quote post Address this user | ||
Hi @Metroplex360, I've just recheck your talkie tour and i've seen new stuff very very interesting as the div video and all the amazon links. Is it still possible to do with last matterport changes ? Or all the stuff you inject in the JavaScript are not available anymore. I'll be really please if you can give us an hint on how to create The same kind of proof of concept I've seen your source and the way you do It but was not able to replicate it. For info I use a different js library for voice speech including also voice recognition I help to debug to integrate it in a domotique platform (jeedom), and that could introduce a whole new way to navigate and in a virtual tour... for example say "show me the master bedroom" , "dollhouse view" clickable text This library works great on google chrome on pc mac and even android. Check the examples and the source and I'm sure you will see the potential If i can test some stuff and help debug for you let me know |
||
Post 28 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
@Olivius sounds like a cool idea! Shoot me a pm. This project is intended as a proof of concept for the time being as there are potential hurdles associated with it. If I had Matterport's blessing, it would allow me the peace of mind to devote more time into shaping this into a community tool. This is presently made possible by running the tour via a local proxy and injecting code. I believe it doesn't violate TOS, but if matterport would like to claim it does, there are many angles they could take. I built this largely as a proof of concept to advocate putting more priority on a public api. |
||
Post 29 IP flag post |
Viewing private msg quote post Address this user | ||
@Olivius @Metroplex360 Talkie is the best thing since sliced bread and it's the future of Matterport. The possibilities are potentially limitless. Please release your API Matterport. Please, please super please with an SDK cherry on top |
||
Post 30 IP flag post |
pirusan private msg quote post Address this user | ||
@Metroplex360 The lady in the bathroom made me laugh!!! | ||
Post 31 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
Quote:Originally Posted by pirusanDude, I'm totally following her on YouTube now. That video is so amazing in so many ways. Did you check the closet? |
||
Post 32 IP flag post |
UserName private msg quote post Address this user | ||
I'm a home buyer exploring a Gear VR living room when I hear the faint sounds of a dog barking somewhere. I follow the sound which gets louder as I move in a particular direction. When I walk into the bedroom where the barking seems to be coming from .. the barking stops abruptly. The room holds no dog but I do see a "picture" of one hanging on the wall. Chalking it all up to a hallucination, I explore the bedroom. But, it only takes a few seconds before I seem to hear the faint sounds of a chainsaw coming from the floor above. Being naturally curious, I head for the stairs .. I may not buy the house, but I sure might have fun exploring it if Gear VR Talkie Tours with directional sound existed |
||
Post 33 IP flag post |
pirusan private msg quote post Address this user | ||
Quote:Originally Posted by Metroplex360 HAHAHAHAHA! Could not catch Snorlax! |
||
Post 34 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
@UserName - Adding a dog barking for fun might be something you'll be able to do soon -- but don't bet on being able to experience this stuff on VR . All these tricks are being done through code injection for the web-based player ... I can't mod an app | ||
Post 35 IP flag post |
UserName private msg quote post Address this user | ||
that's ok .. before we had VR headsets, all we had was computer monitors and Web browsers. I looked at the code and that's interesting what you did with the lastPano detection. The whole thing is still impressive even without directional sound. If you could create a GUI front end that generated that JS code, Matterport fans might have a field day choosing their own custom sounds(barks/chainsaws/voice-overs) and room locations where the sounds play. | ||
Post 36 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
Super cool idea, @UserName!! We are scrambling to keep up with all of the great ideas coming this way, but we're reading all of them. More soon! :-) | ||
Post 37 IP flag post |
Viewing private msg quote post Address this user | ||
@Metroplex360 Front the respective website as an app? Also I was wondering, is the media called as you walk through or is the media contained and pre-loaded within the tour? | ||
Post 38 IP flag post |
UserName private msg quote post Address this user | ||
I also had @Baxter's idea too but in a different way. People install games and programs they love and need. Maybe they'd physically install a Windows .exe version of something like this if that program could work with any Matterport model and did things a regular Matterport model could not. There's nothing to stop anyone from putting a Web browser, like MS's Webbrowser control, inside a desktop program. Benefits of that are 1) The desktop program can communicate with the model using JS the same way you do and 2) You gain all the benefits that come from using a Windows program. Essentially you have a hybrid experience -- a Web app running inside a desktop program. You can then do anything a Windows program can do, such as playing directional sound or accessing the hard drive, accessing a database and overlaying your own user interface elements. The desktop app can even send keystrokes and mouse movements to the embedded Web browser control, grab screen shots of any part of the model as needed, etc. Evernote's one example of a "Web" company that decided we needed a desktop version of their Web app too. I've done lots of these hybrid Web/Desktop applications over the years. When an HTML document exposes important global variables and we understand the DOM, amazing things can happen when a desktop app plays with all that information. @Baxter -- that's probably not exactly what you meant by "app." I assume you mean make your own Web-based app/site that does what your demo does now. |
||
Post 39 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
@Baxter Matterport Showcase uses some amazing quaternian mathematics to calculate which tours are nearby and which is the likely candidate to move to when using arrow keys and clicking. It also uses these calculations to figure what it should preload. When we start the tour, it's loading all of the mesh textures and some initial cube faces. From there, the tour keeps preloading nearby scenes until it's done. Matterport added the &qs=1 parameter recently that I believe skips the mesh load and loads that after the initial panoramas. Hope that was accurate enough and helped |
||
Post 40 IP flag post |
Viewing private msg quote post Address this user | ||
Yeah it did help thanks bud, but it also led to an hour of brain frazzle trying to get a basic understanding of quaternions So are you somehow attaching actions to a certain radius from an x,y,z location within the tour? I understood the premise of what you meant before but think I have a better understanding now. But wouldn't this get tricky if you needed to add a load of different points within a close proximity to one another? It'd be nice to bring up a sidebar of multiple options for the user to scroll thru to get around this The reason I ask about called externally or pre-loaded was related to the time it would take a media rich pre-loaded tour to load and would a user be prepared to wait for however long it may take. @UserName It would be amazing to see what you've described put into practice with an MP tour. I like your version a lot better than what I originally meant by app. I see real importance in also tying this tech into apps for smart televisions as house searching is (generally) a shared experience. MP has the potential to completely dominate the real life based VR experience world if it does two things: Develop a camera that works in daylight (perhaps not so easy) and release its APIs (easy) and leave the developer community to do amazing things with their tech. |
||
Post 41 IP flag post |
Frisco, Texas |
Metroplex360 private msg quote post Address this user | |
@Baxter - Matterport built showcase with an internal API that their developers use. It's basically an object with classes and functions -- it's what any sophisticated platform would have. The Internal API can immediately return your X, Y, Z location within a tour -- however -- it can also return the pano id of what you are looking at. Since we are always moving between panos, that's much easier. Matterport's killer feature is that it is a camera that automatically shoots fully stitched 360s and generates a tour with a dense number of panoramas that are all connected. Their showcase platform is phenominal. Any concerns that the company would have concerning users utilizing their scan data and images for other purposes are unfounded in my opinion as they still sold a camera, processing/hosting fees, and control the best way to view Matterport Tours. |
||
Post 42 IP flag post |
This topic is archived. Start new topic?