Hi,<br><br>just to know whether it is the problem of samba re-export, instead of re-exporting the client mount point through samba, can you re-export it through glusterfs itself and try whether you still find the problem?<br>
<br>regards,<br><br><div class="gmail_quote">On Wed, May 6, 2009 at 3:23 AM, <span dir="ltr"><<a href="mailto:m.c.wilkins@massey.ac.nz">m.c.wilkins@massey.ac.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Hi,<br>
<br>
Using samba on top of gluster, and when one brick goes down I can't<br>
see files that are actively being made on functional bricks.<br>
<br>
Running 2.0.0, I have 2 machines in a NUFA arrangement (log below),<br>
gluster mounted on /mnt, sharing /mnt out from one node via samba. On<br>
the machine that is doing the samba I do a mkdir /mnt/me, then start<br>
copying a big file into /mnt/me. I can do a ls -l /mnt/me and see the<br>
file growing in there. I take down my second node (umount /mnt), back<br>
on the first node everything is fine, the copy continues since the<br>
file is going to a local brick anyway. I can do a ls -l /mnt/me and<br>
see the file growing there fine.<br>
<br>
I repeat the above but I'm copying a file into /mnt/me via samba.<br>
When the second node goes down, the copy continues, but I can't see<br>
the file in /mnt/me (a ls -l /mnt/me/nameoffile shows it, but a ls -l<br>
/mnt/me doesn't). If I start a new copy while the second node is<br>
down, then a ls -l /mnt/me shows that file fine. It is only files<br>
that are in the process of being copied while the second node goes<br>
down, and only under samba, that I see the problem.<br>
<br>
I thought it might be something to do with samba preallocating the<br>
entire file, then filling it up during the copy. I set 'strict<br>
allocate = yes' in my smb.conf, which doesn't stop the preallocating<br>
but it does make the preallocation real slow, slow enough that i can<br>
watch the file growing and then take the second node down, but then<br>
wham it is then not visible via a ls -l on its directory.<br>
<br>
It might not sound like a huge issue, since the copy works. But I am<br>
currently testing our gluster and demonstrating to a user that it will<br>
work OK in the situation when a brick other than actively used one on<br>
the samba server goes down.<br>
<br>
Please, any help much appreciated.<br>
<br>
Matt<br>
<br>
---<br>
Version : glusterfs 2.0.0 built on May 6 2009 09:02:18<br>
TLA Revision : 7b2e459db65edd302aa12476bc73b3b7a17b1410<br>
Starting Time: 2009-05-06 09:05:19<br>
Command line : /usr/local/sbin/glusterfs --log-level=NORMAL --volfile=/usr/local/etc/glusterfs/glusterfsd3.vol /mnt<br>
PID : 20866<br>
System name : Linux<br>
Nodename : mu-rhdev1<br>
Kernel Release : 2.6.18-128.1.6.el5<br>
Hardware Identifier: x86_64<br>
<br>
Given volfile:<br>
+------------------------------------------------------------------------------+<br>
1: volume posix0<br>
2: type storage/posix<br>
3: option directory /export/brick0<br>
4: end-volume<br>
5:<br>
6: volume locks0<br>
7: type features/locks<br>
8: subvolumes posix0<br>
9: end-volume<br>
10:<br>
11: volume brick0<br>
12: type performance/io-threads<br>
13: subvolumes locks0<br>
14: end-volume<br>
15:<br>
16: volume posix1<br>
17: type storage/posix<br>
18: option directory /export/brick1<br>
19: end-volume<br>
20:<br>
21: volume locks1<br>
22: type features/locks<br>
23: subvolumes posix1<br>
24: end-volume<br>
25:<br>
26: volume brick1<br>
27: type performance/io-threads<br>
28: subvolumes locks1<br>
29: end-volume<br>
30:<br>
31: volume server<br>
32: type protocol/server<br>
33: option transport-type tcp<br>
34: option auth.addr.brick0.allow *<br>
35: option auth.addr.brick1.allow *<br>
36: subvolumes brick0 brick1<br>
37: end-volume<br>
38:<br>
39: volume mu-rhdev1-0<br>
40: type protocol/client<br>
41: option transport-type tcp<br>
42: option remote-host mu-rhdev1<br>
43: option remote-subvolume brick0<br>
44: end-volume<br>
45:<br>
46: volume mu-rhdev1-1<br>
47: type protocol/client<br>
48: option transport-type tcp<br>
49: option remote-host mu-rhdev1<br>
50: option remote-subvolume brick1<br>
51: end-volume<br>
52:<br>
53: volume mu-rhdev2-0<br>
54: type protocol/client<br>
55: option transport-type tcp<br>
56: option remote-host mu-rhdev2<br>
57: option remote-subvolume brick0<br>
58: end-volume<br>
59:<br>
60: volume nufa<br>
61: type cluster/nufa<br>
62: option local-volume-name mu-rhdev1-0<br>
63: subvolumes mu-rhdev1-0 mu-rhdev1-1 mu-rhdev2-0<br>
64: end-volume<br>
65:<br>
66: volume writebehind<br>
67: type performance/write-behind<br>
68: option cache-size 1MB<br>
69: subvolumes nufa<br>
70: end-volume<br>
71:<br>
72: # before or after writebehind?<br>
73: volume ra<br>
74: type performance/read-ahead<br>
75: subvolumes writebehind<br>
76: end-volume<br>
77:<br>
78: volume cache<br>
79: type performance/io-cache<br>
80: option cache-size 512MB<br>
81: subvolumes ra<br>
82: end-volume<br>
83:<br>
<br>
+------------------------------------------------------------------------------+<br>
2009-05-06 10:33:15 N [glusterfsd.c:1152:main] glusterfs: Successfully started<br>
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused)<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected<br>
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-1: connection failed (Connection refused)<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-1: disconnected<br>
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused)<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected<br>
2009-05-06 10:33:15 E [socket.c:743:socket_connect_finish] mu-rhdev1-0: connection failed (Connection refused)<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected<br>
2009-05-06 10:33:15 N [client-protocol.c:7014:notify] mu-rhdev1-0: disconnected<br>
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1014" target="_blank">130.123.131.13:1014</a><br>
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1017" target="_blank">130.123.131.13:1017</a><br>
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1016" target="_blank">130.123.131.13:1016</a><br>
2009-05-06 10:33:16 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1015" target="_blank">130.123.131.13:1015</a><br>
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.2:1019" target="_blank">130.123.131.2:1019</a><br>
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0'<br>
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded<br>
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.2:1018" target="_blank">130.123.131.2:1018</a><br>
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-0: attaching to the local volume 'brick0'<br>
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-0: connection and handshake succeeded<br>
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.2:1017" target="_blank">130.123.131.2:1017</a><br>
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1'<br>
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded<br>
2009-05-06 10:33:25 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.2:1016" target="_blank">130.123.131.2:1016</a><br>
2009-05-06 10:33:25 W [client-protocol.c:6320:client_setvolume_cbk] mu-rhdev1-1: attaching to the local volume 'brick1'<br>
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev1-1: connection and handshake succeeded<br>
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded<br>
2009-05-06 10:33:25 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded<br>
2009-05-06 10:33:29 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: <a href="http://130.123.131.13:1014" target="_blank">130.123.131.13:1014</a> disconnected<br>
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: <a href="http://130.123.131.13:1015" target="_blank">130.123.131.13:1015</a> disconnected<br>
2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-1<br>
2009-05-06 10:33:29 E [socket.c:102:__socket_rwv] server: readv failed (Connection reset by peer)<br>
2009-05-06 10:33:29 E [socket.c:562:__socket_proto_state_machine] server: read (Connection reset by peer) in state 1 (<a href="http://130.123.131.13:1016" target="_blank">130.123.131.13:1016</a>)<br>
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: <a href="http://130.123.131.13:1016" target="_blank">130.123.131.13:1016</a> disconnected<br>
2009-05-06 10:33:29 N [server-protocol.c:8272:notify] server: <a href="http://130.123.131.13:1017" target="_blank">130.123.131.13:1017</a> disconnected<br>
2009-05-06 10:33:29 N [server-helpers.c:723:server_connection_destroy] server: destroyed connection of mu-rhdev2-7959-2009/05/06-10:33:16:708168-mu-rhdev1-0<br>
2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)<br>
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:38 E [socket.c:743:socket_connect_finish] mu-rhdev2-0: connection failed (Connection refused)<br>
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:38 N [client-protocol.c:7014:notify] mu-rhdev2-0: disconnected<br>
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1009" target="_blank">130.123.131.13:1009</a><br>
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1016" target="_blank">130.123.131.13:1016</a><br>
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1011" target="_blank">130.123.131.13:1011</a><br>
2009-05-06 10:33:43 N [server-protocol.c:7502:mop_setvolume] server: accepted client from <a href="http://130.123.131.13:1010" target="_blank">130.123.131.13:1010</a><br>
2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded<br>
2009-05-06 10:33:45 N [client-protocol.c:6327:client_setvolume_cbk] mu-rhdev2-0: connection and handshake succeeded<br>
<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>Raghavendra G<br><br>