matterport-dl in Github (an open source matterport downloader)17967
Pages:
1
|
fdd private msg quote post Address this user | |
| matterport-dl has someone tried it? |
||
| Post 1 • IP flag post | ||
 WGAN3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| Thank you @fdd A two years old project about "Matterport Download" but no one here mentioned it before today. It is not normal that WGAN is the largest forum of MSPs. matterport-dl does download lots of data, but I can't open the result in local server page. Maybe I am using wrong. But all the tiles data were downloaded and they can be used to make 16K x 8K panorama for sure. BTW I found this below project from matterport-dl. https://github.com/fdd4s/matterport-downloader |
||
| Post 2 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| The last ( "matterport-downloader" ) is mine @JuMP but far from all the options of "matterport-dl". "matterport-downloader" is just to grab pano skybox images from matterport. And it can be converted to equirectangular standard panos images with "cube2sphere" open source software (explained in the README of the project). |
||
| Post 3 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| The result of open source "cube2sphere" is not good enough. Its high quality 16K x 8K output will be very big. We should have the best quality with the smallest JPEG result. The interpolation algorithm from 6 x 4K x 4K cube to 16K x 8K sphere and the compress algorithm of JPEG are two keys. krpano is good at cube2shpere and JPEG encode, pay 159€ lic you will get a very good cube2sphere convertor. Webp format can be used also, it will make the result smaller. We always use Webp format in our projects. |
||
| Post 4 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| cube2sphere is just a script for Blender, that automatizes the conversion, you can modify it and increase the size of the render so much as you want. But keep in mind that matterport faces of cubefaces/skybox are 1024x1024 pixels and in JPEG format. By the other side, a equirectangular JPEG with the right JFIF tags is playable in several pano viewers, including some smartphones natively, without install apps. |
||
| Post 5 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| @fdd Yes, the open source "cube2sphere" uses blender render, that is why his result is not good and take long time to generate. 3D render engine is powerful, easy to use without math, but not for the fast accurate convertion. The better way is using the best interpolation algorithm with math ( most time it is triangle functions ) to calculate the result pixel by pixel, leveling (3 rotations along with X,Y and Z axis) can also be done at the same time with more complex triangle transformation. And multithread can speed up the average processing on a server machine with enough memory. The maximum speed for our server is 30 seconds for one 16K x 8K generation with leveling process average. We do it with 12 thread on our 2 x Xeon cpu server, each single thread costs 4GB memory and 6 mins for one 16K x 8K result, no 3D graphcard required only cpu calculation is ok. BTW the cubemap from Matterport can be 4K x 4K in JPEG, so that we can make 16K x 8K panorama result. As for JFIF information, it can be added automatically during the processing. We used to add it, so that our client can download panorama at any points of the tour and view it in 360 mode directly in facebook pages or other applications who support JFIF with equirectangular information. |
||
| Post 6 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| Certainly it's slow blender/cube2sphere, but about quality issue, I think it's not the problem of Blender. All equirectangular images seems to loss quality if you compare to skybox of similar 6x size, due to how a equirectangular image works, but that can be solved increasing render size so much as you want. Once you have the equirectangular image, add the JFIF tags is easy, and I recommend open source software too, this tool: https://github.com/exiftool/exiftool |
||
| Post 7 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| @fdd How about we compare the quality of the cube2sphere between Blender's 3D render and my calculation with interpolation algorithm? You select one input from any Matterport showcase's point. You use blender and I use my code. We post the results here with the processing seconds used. Thank you. |
||
| Post 8 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
Quote:Originally Posted by JuMP Good idea, but select you the Matterport view of house you want. The processing seconds depend of the machine that run the code, not only of how good is the code. But as I said, cube2sphere is very slow. Although I think is not something important, because you only do it once and that's all. But I'd like to compare the quality, I can see no difference between a matterport official viewer and a Blender render with cube2sphere, I see the same quality if the size is big enough. |
||
| Post 9 • IP flag post | ||
|
mori private msg quote post Address this user | |
| @fdd - yes and it worked for me - tried it about 8-12 month ago. unfortunately it does not fetch the preview images for tour highlights and maybe also other data I haven´t tried out. |
||
| Post 10 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| I uploaded a script to download 360 panos from Google Street View if anyone is interested. The script adds automatically the exif pano tags to make the image viewable in several 360 players, like Ricoh Theta. https://github.com/fdd4s/streetview-dl Google stores his street views images in equirectangular format, so it's not needed to a slow cube2sphere conversion, just to mount the equirectangular tiles in a single image (google splits the pano in hundreds tiles to load only the part that user request).  |
||
| Post 11 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| @fdd Let us compare the quality of this pano: https://we.tl/t-EFeQLYkyOt (the download link of 16K x 8K picture)  It is the first view of this Matterport DEMO for Pro2. https://my.matterport.com/show/?m=Re2dgaSVCRj |
||
| Post 12 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| Here is my conversion @JuMP https://we.tl/t-M2WRmG5eA5 I took 2 panos, the one you said and another (the first that appears in internal matterport catalog of that house). I took as source 4K skybox (just to change one line to my script, that by default it downloads 1K skybox, .php line 12 "high" replaced by "/4k/" ). Exif equirectangular pano tags added, so it can be viewed in a compatible viewer like Ricoh Theta for Android. I made the render of 16384x8192 with cube2sphere 100% quality JPG, and then I resized it to 8192x4096 75% quality JPG in other files (pano1.jpg and pano2.jpg). Running time in a old machine with Linux Ubuntu (i7 3.2 GHz 4 Cores 24 GB RAM): The same pano than you: 697 seconds The other pano: 799 seconds The time is cube2sphere render + imagemagick format conversion from PNG to JPG 100% quality.   |
||
| Post 13 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| This is the result of image comparision using ImageMagick https://imagemagick.org/script/compare.php compare -verbose -metric mae Re2dgaSVCRj-Pano01.jpg pan-4k-24d4a3b4753d41ab85e47a8fac981bd6-equi.jpg difference.png Re2dgaSVCRj-Pano01.jpg JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 49.2MB 4.590u 0:04.600 pan-4k-24d4a3b4753d41ab85e47a8fac981bd6-equi.jpg JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 47.45MB 5.600u 0:05.620 Image: Re2dgaSVCRj-Pano01.jpg Channel distortion: MAE red: 300.964 (0.00459242) green: 280.606 (0.00428177) blue: 293.374 (0.00447659) all: 291.648 (0.00445026) i.e: difference between two images is 0.45%, the two images are the same in a 99.55% By the other side, before we going crazy with the highest quality, we must remember: Human eyes have 6 millions cone cells for red/green/blue colors, i.e: our eyes are like 2 MP cameras when we stare to a fix point. https://en.wikipedia.org/wiki/Cone_cell If you dont plan to do zooms, and just to view the pano in a VR headset, maybe 8K width pano and 75% quality JPG is more than enough. |
||
| Post 14 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| @fdd Thank you for your result and information. Glad to see the quality is similar. I don't know how ImageMagick makes the comparision. But from below picture, I can see the 0.45% difference with my naked eye:  About the speed: 697 seconds for one 16K x 8K is a little slow. With this speed only single generation can be processed. It is not possible to make bulk process. The system that I am using is a ten years old server. It has dual 8 Core Xeon E5-2690 @ 2.9GHz with 64GB Memory. And it can handle ten 16K x 8K generation in parallel within one minute. So the average process time is 6 seconds for each one. We use Matterport to make digital clone of many different spaces. I think zoom view is very important for some projects so that we can see the detail, the higher resolution is better. Even 16K x 8K is not enough, but it is the limit that Matterport can provide now. BTW the leveling process (rotate the panorama and make them horizontal and toward the same direction) is more difficult than the generation of a 16K x 8K panorama, more calculations were required(it is a per-pixel / per-channel processing of 16K x 8K x RGB). Our server can handle it in parallel mode and the average process time of leveling is 30 seconds for each 16K x 8K panorama. |
||
| Post 15 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| @JuMP you're right, the left image is clearly sharp-edged and the right is smooth. But it's needed a zoom to view it. About the speed: I still think it's not very important, to make the conversion I use a script that batch the conversion of all panos of a single house, it's an automatic job, slow but automatic. I will publish it when I'll make some changes (my current code is a little dirty, I reuse to different tasks). The use of equirectangular is more a topic about achieve wide compatibility and easy to deliver (one file one pano) among pano viewers than from quality. Also if you want to use 360 panos images in a 360 video, the frames of a 360 youtube video are in equirectangular format. If you want to achieve the higher quality then the best is not conversion at all and just use the 4k skybox from matterport (webgl three.js for chrome/firefox and panini for linux/windows allow view skybox cubefaces panos). BTW I invite you to this thread about ideas of best ways to view downloaded panos: https://www.wegetaroundnetwork.com/topic/17965/page/1/options-for-tour-deployment-in-vr-ie-devices--offline/ |
||
| Post 16 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
BTW here is the object of the previous example, taken from the skybox original image, no changes: I uploaded the full original 4k skybox of that pano here: https://we.tl/t-ovbSR4jTgM I think ImageMagick comparision works well, most of two images are identical, and that object is 150 pixels width in a image of 16K width, with tiny differences. 10% of differences in that object is less than 0.1% in whole image. You must keep in mind: image filters can give the sensation of higher quality, but are filters and it's not the real image. Just increase contrast in most pics can give the sensation of higher definition. And if that is the preference I think it's better use open source software to apply those filters, knowing how works and with the possibility of future changes. |
||
| Post 17 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| @fdd About which one is better between the equirectangular pano and the skybox cubemap, I would like to select the equirectangular pano. In below picture: If you divide the circumference equally, the equirectangular pano has less stretching and more nature than a skybox cube.  The skybox cube waste more pixels at the corner of each direction and can't provide the same physical resolution in the 360 degree horizontal direction, the benefit of skybox map is it looks familiar to human eyes and skybox cube has much less faces than a sky sphere. Another benefit of the equirectangular pano is more easier for retouch. It has only one seam compare to the skybox cubemap has six seams. The more seams make people nervous when they need to retouch the picture. ----------------- About the image quality and filter: Please refer below picture. I add the original part from the 4K cubemap and all of them are in actual size.  The second line is the result after enhance filter that powered by open-source software "ImageMagick". |
||
| Post 18 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| This is a clear difference between use or not to use filters, but personally I can't say what's better. I think the best is the more like the real world experience, but all we know examples of sellers that prefer nice pics of their products than real pics of their products. About the use of cubefaces/skybox or equirectangular: You must keep in mind that choice in matterport is different, because matterport internal public images are in skybox format, so if you want to save matterport pics losslessly, then the better choice is always (at least) skybox, without changes. In the same way, if you want to store Google Street View panos images then it's better equirectangular, because original Google Street View 360 panos are in equirectangular format. Another thing would be if you take the 360º images with another cameras or if you are the owner of the matterport house and they offer to you download the panos in a highest quality that is not publicly available when you see the house with the public url. In that case, maybe equirectangular is better choice. My general opinion for matterport is: -Always save the original skybox. -To make viewable the pano easily for wide kind of users: equirectangular JPG with exif 360 tags and 8K width (larger pics could be unopenable in some viewers). -If you want to maximize compatibility of panos among users, then make a slideshow 360º youtube video using equirectangular panos as frames, several seconds by each frame. It can be done with FFmpeg and a google free tool to add 360 tags to video and make it uploadable to youtube. Almost all devices to view pano images support 360 youtube videos. In the case of youtube 360º videos it can be used higher widths of 8K and compatibility is keep, because youtube automatically resize the video to smaller sizes. AFAIK the most secure, compatible and known choice to get free hosting of 360 panos is a youtube 360º video. And a 360º youtube video is just a MP4/webm with a sequence of equirectangular frames and a special 360º video tag. |
||
| Post 19 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| I've discovered a new free tool to make the conversion from skybox cubemap to equirectangular faster than cube2sphere. The v360 filter of FFmpeg https://ffmpeg.org/ffmpeg-filters.html#v360 I've implemented a script (like cube2sphere) to use it easily with images, here: https://github.com/fdd4s/skybox2equirectangular (only linux) The conversion of the previous example of this thread is made in 25 seconds (instead 697 of cube2sphere). My scripts besides adds automatically exif tags of 360 equirectangular pano. 25 seconds includes the image formats conversions and exif tags added, but the stitched process only of skybox-equirectangular conversion is around 16 seconds. I've done some changes to my matterport download script (it's faster now, it selects automatically the highest skybox quality, and retry downloads 3 times if it fails) and I've merged both projects in one: https://github.com/fdd4s/matterport-downloader (original project, skybox download, Windows & Linux) https://github.com/fdd4s/matterport-dl-equi (merged project, skybox and equirectangular automatic conversion, only Linux) The complete download and conversion of the matterport house example in this thread is done in 42 minutes (40 equirectangular panos, an average of 1 minute per pano). That includes 4k skybox download, conversion to 16K width equirectangular pano, and a resize to 8K width equirectangular pano, with exif 360 tags included). It could be even faster implementing download using aria2 (like my streetview-dl script) instead curl, but I've not done it yet.  The comparision of equirectangular panos with imagemagick is the next: compare -verbose -metric mae Re2dgaSVCRj-Pano01.jpg ffmpeg.jpg difference1.png Re2dgaSVCRj-Pano01.jpg JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 49.2MB 4.530u 0:04.550 ffmpeg.jpg JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 48.57MB 5.400u 0:05.419 Image: Re2dgaSVCRj-Pano01.jpg Channel distortion: MAE red: 388.74 (0.00593179) green: 357.336 (0.0054526) blue: 371.854 (0.00567413) all: 372.643 (0.00568617) Re2dgaSVCRj-Pano01.jpg=>difference1.png JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 63.28MB 55.440u 2:31.259 ***** compare -verbose -metric mae cube2sphere.jpg ffmpeg.jpg difference2.png cube2sphere.jpg JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 47.45MB 4.530u 0:04.550 ffmpeg.jpg JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 48.57MB 5.370u 0:05.379 Image: cube2sphere.jpg Channel distortion: MAE red: 181.381 (0.0027677) green: 171.599 (0.00261844) blue: 186.645 (0.00284803) all: 179.875 (0.00274472) writing raw profile: type=exif, length=110 writing raw profile: type=xmp, length=3678 cube2sphere.jpg=>difference2.png JPEG 16384x8192 16384x8192+0+0 8-bit sRGB 67.46MB 58.340u 2:28.040 Differences about @JuMP pano and ffmpeg pano: 0.6% Differences about cube2sphere pano and ffmpeg pano: 0.3% About imagemagick comparision percentage: it's based in pixels colors, i.e: if you compare a plain black image and a plain white image it will be 100% of differences, but if you compare a red and a orange plain images, differences will be less. And if you compare two random completely differents real pics of the same size, the average percetange of differences will be of 25%, so to get a more realistic view of differences, the percentage should be multiplied by a factor of 4. |
||
| Post 20 • IP flag post | ||
|
WGAN 3rd Party Service Member Beijing |
JuMP private msg quote post Address this user | |
| @fdd Try multithread process in parallel, speed could be 10x faster. | ||
| Post 21 • IP flag post | ||
|
|
fdd private msg quote post Address this user | |
| Thanks for the advice @JuMP. In cube2sphere (blender) there's a increase of speed with --threads option (double, maybe it could be higher in machines with higher number of cores, mine is just 4 cores). In FFmpeg and ImageMagick I see no increase, I think it's already optimized to use one thread per core by default. (FFmpeg option -threads, and ImageMagick option thread in policy.xml). And running several instances of the script in several houses, the conversion gets slower as expected. BTW I've just fixed a little bug that made the script unusable in last versions of PHP, now it works in PHP 8 too. |
||
| Post 22 • IP flag post | ||
Pages:
1This topic is archived. Start new topic?