<div dir="ltr"><div>Dear Xavier,</div><div>I realized that the volume was not build properly when doing the first analyses suggested by Davide, but I'm sure that this is not the problem and so I quickly dismissed it. Also, we need a replica but not so strictly, maybe in the future with the next volume I'll build it properly. Anyway, yes, the volume got birth on "pedrillo" with a replica-2 and the next day was expanded on "osmino", again with replica-2, just by adding bricks and doing a rebalance, that was just tried. I'm saying "tried" because it got "stuck", consuming a lot of RAM (almost all, 16 GB), and it was counting million of files that I think don't even exist on the volume, so I stopped it. Do you think that it might be worth restarting?</div>
<div>The brick logs are there, just before your nice reply asking for them. They're all the same, except for the order of the connection, all seem very good.</div><div>Yes, you're right, many files are missing through the mounpoint, I just choose one. But then again, what do you think about the fact that ls hides them but calling them from the mount point are not counted as "file not found"? "leggi_particelle" was working calling it from the bricks and from the mountpoint, even if ls didn't show it in the mountpoint...</div>
<div>And here it is all the other info that you were asking: first of all a collection of ls of the folder on all the bricks of the volume (10)</div><div> </div><div>/storage/1/data/stefano/leggi_particelle:<br>total 20<br>
drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 04:58 ../<br>-rwxr-xr-x 2 stefano user 286 Feb 25 17:24 Espec.plt*<br>lrwxrwxrwx 2 stefano user 53 Feb 13 11:30 parametri.cpp -> /tamino/stefano/codice/leggi_particelle/parametri.cpp<br>
drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/</div><div> </div><div>/storage/2/data/stefano/leggi_particelle:<br>total 20<br>drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 04:58 ../<br>
-rwxr-xr-x 2 stefano user 286 Feb 25 17:24 Espec.plt*<br>lrwxrwxrwx 2 stefano user 53 Feb 13 11:30 parametri.cpp -> /tamino/stefano/codice/leggi_particelle/parametri.cpp<br>drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/</div>
<div> </div><div>/storage/5/data/stefano/leggi_particelle:<br>total 892<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../<br>lrwxrwxrwx 2 stefano user 50 Apr 11 19:20 filtro.cpp -> /tamino/stefano/codice/leggi_particelle/filtro.cpp<br>
lrwxrwxrwx 2 stefano user 70 Apr 11 19:20 leggi_binario_ALaDyn_fortran.h -> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.h<br>-rwxr-xr-x 2 stefano user 705045 May 22 18:24 leggi_particelle*<br>
-rwxr-xr-x 2 stefano user 61883 Dec 16 17:20 leggi_particelle.old01*<br>-rwxr-xr-x 2 stefano user 106014 Apr 11 19:20 leggi_particelle.old03*<br>---------T 2 root root 0 May 24 17:16 parametri.cpp<br>drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/</div>
<div> </div><div>/storage/6/data/stefano/leggi_particelle:<br>total 892<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../<br>lrwxrwxrwx 2 stefano user 50 Apr 11 19:20 filtro.cpp -> /tamino/stefano/codice/leggi_particelle/filtro.cpp<br>
lrwxrwxrwx 2 stefano user 70 Apr 11 19:20 leggi_binario_ALaDyn_fortran.h -> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.h<br>-rwxr-xr-x 2 stefano user 705045 May 22 18:24 leggi_particelle*<br>
-rwxr-xr-x 2 stefano user 61883 Dec 16 17:20 leggi_particelle.old01*<br>-rwxr-xr-x 2 stefano user 106014 Apr 11 19:20 leggi_particelle.old03*<br>---------T 2 root root 0 May 24 17:16 parametri.cpp<br>drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/<br>
</div><div>/storage/arc1/data/stefano/leggi_particelle:<br>total 144<br>drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../<br>lrwxrwxrwx 2 stefano user 53 Feb 22 19:40 binnaggio.cpp -> /tamino/stefano/codice/leggi_particelle/binnaggio.cpp<br>
-rwxr-xr-x 2 stefano user 350 Feb 25 17:24 Etheta.plt*<br>lrwxrwxrwx 2 stefano user 72 Mar 22 2012 leggi_binario_ALaDyn_fortran.cpp -> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.cpp<br>
lrwxrwxrwx 2 stefano user 55 Mar 22 2012 leggi_campi.cpp -> /tamino/stefano/codice/leggi_particelle/leggi_campi.cpp<br>lrwxrwxrwx 2 stefano user 60 Apr 11 19:20 leggi_particelle.cpp -> /tamino/stefano/codice/leggi_particelle/leggi_particelle.cpp<br>
-rwxr-xr-x 2 stefano user 97536 Mar 25 12:46 leggi_particelle.old02*<br>-rwxr-xr-x 2 stefano user 923 May 12 12:42 plot_den.plt*<br>lrwxrwxrwx 2 stefano user 54 Mar 22 2012 swap_tools.cpp -> /tamino/stefano/codice/leggi_particelle/swap_tools.cpp<br>
drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/<br>-rwxr-xr-x 2 stefano user 309 Feb 25 17:24 xpx.plt*</div><div> </div><div>/storage/arc2/data/stefano/leggi_particelle:<br>total 144<br>drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./<br>
drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../<br>lrwxrwxrwx 2 stefano user 53 Feb 22 19:40 binnaggio.cpp -> /tamino/stefano/codice/leggi_particelle/binnaggio.cpp<br>-rwxr-xr-x 2 stefano user 350 Feb 25 17:24 Etheta.plt*<br>
lrwxrwxrwx 2 stefano user 72 Mar 22 2012 leggi_binario_ALaDyn_fortran.cpp -> /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.cpp<br>lrwxrwxrwx 2 stefano user 55 Mar 22 2012 leggi_campi.cpp -> /tamino/stefano/codice/leggi_particelle/leggi_campi.cpp<br>
lrwxrwxrwx 2 stefano user 60 Apr 11 19:20 leggi_particelle.cpp -> /tamino/stefano/codice/leggi_particelle/leggi_particelle.cpp<br>-rwxr-xr-x 2 stefano user 97536 Mar 25 12:46 leggi_particelle.old02*<br>-rwxr-xr-x 2 stefano user 923 May 12 12:42 plot_den.plt*<br>
lrwxrwxrwx 2 stefano user 54 Mar 22 2012 swap_tools.cpp -> /tamino/stefano/codice/leggi_particelle/swap_tools.cpp<br>drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/<br>-rwxr-xr-x 2 stefano user 309 Feb 25 17:24 xpx.plt*<br>
</div><div>/storageOsmino/1/data/stefano/leggi_particelle:<br>total 24<br>drwxr-xr-x 3 stefano user 4096 May 29 05:00 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 04:55 ../<br>---------T 2 root root 0 May 29 05:00 leggi_particelle<br>
drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/</div><div> </div><div>/storageOsmino/2/data/stefano/leggi_particelle:<br>total 24<br>drwxr-xr-x 3 stefano user 4096 May 29 05:00 ./<br>drwxr-xr-x 14 stefano user 4096 May 28 04:55 ../<br>
---------T 2 root root 0 May 29 05:00 leggi_particelle<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/</div><div> </div><div>/storageOsmino/4/data/stefano/leggi_particelle:<br>total 16<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./<br>
drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/</div><div> </div><div>/storageOsmino/5/data/stefano/leggi_particelle:<br>total 16<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./<br>
drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../<br>drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/<br></div><div>=======================================</div><div>=======================================</div><div>and these are the attributes of the folders in every brick</div>
<div> </div><div># file: storage/1/data/stefano/leggi_particelle<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x00000001000000000000000033333332</div><div> </div><div># file: storage/2/data/stefano/leggi_particelle<br>
trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x00000001000000000000000033333332</div><div> </div><div># file: storage/5/data/stefano/leggi_particelle<br>trusted.afr.data-client-2=0x000000000000000000000000<br>
trusted.afr.data-client-3=0x000000000000000000000000<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x00000001000000003333333366666665</div><div> </div><div># file: storage/6/data/stefano/leggi_particelle<br>
trusted.afr.data-client-2=0x000000000000000000000000<br>trusted.afr.data-client-3=0x000000000000000000000000<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x00000001000000003333333366666665</div>
<div> </div><div># file: storage/arc1/data/stefano/leggi_particelle<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x0000000100000000ccccccccffffffff</div><div> </div><div># file: storage/arc2/data/stefano/leggi_particelle<br>
trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x0000000100000000ccccccccffffffff<br></div><div># file: storageOsmino/1/data/stefano/leggi_particelle<br>trusted.afr.data-client-6=0x000000000000000000000000<br>
trusted.afr.data-client-7=0x000000000000000000000000<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x00000001000000006666666699999998</div><div> </div><div># file: storageOsmino/2/data/stefano/leggi_particelle<br>
trusted.afr.data-client-6=0x000000000000000000000000<br>trusted.afr.data-client-7=0x000000000000000000000000<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x00000001000000006666666699999998</div>
<div> </div><div># file: storageOsmino/4/data/stefano/leggi_particelle<br>trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x000000010000000099999999cccccccb</div><div> </div><div># file: storageOsmino/5/data/stefano/leggi_particelle<br>
trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105<br>trusted.glusterfs.dht=0x000000010000000099999999cccccccb<br></div><div>===========================================</div><div>Something is not consistent here and I can see it by myself...</div>
<div>I hope to be forgiven for all these details and spamming in the mailing list. I'm just arrived and I'm already filling it up... Sorry. But also thanks a lot for your help!</div><div> </div><div> Stefano</div>
<div class="gmail_extra"><br> </div></div>