<div dir="ltr">An hour of googling didn't turn up the answer, so I'll ask here: do you know about when the linux kernel changed to enable inode64 by default? I haven't run into the issue Pat had, but I don't want to! <div>
<br></div><div>I wish inode64 use were reported by xfs_info or something.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Feb 6, 2014 at 10:57 AM, Brian Foster <span dir="ltr"><<a href="mailto:bfoster@redhat.com" target="_blank">bfoster@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 02/06/2014 11:25 AM, Pat Haley wrote:<br>
><br>
> Hi Brian,<br>
><br>
</div><div class="im">> gluster-0-1 did not recognize the delaylog option,<br>
> but when I mounted the disk with nobarrier,inode64<br>
> I was able to write to the disk both directly<br>
> and from a client through gluster.<br>
><br>
> Assuming inode64 was the key, was the problem<br>
> that XFS could not address the inodes withour<br>
> 64 bit representation? Just curious.<br>
><br>
> Problem solved! Thanks!<br>
><br>
<br>
</div>Oh, ok. That's good to hear. inode64 tends to slip my mind because it's<br>
fairly standard at this point. It's enabled by default on more recent<br>
kernels.<br>
<br>
Effectively, the problem is as you describe. Without inode64 support,<br>
inodes are restricted to the first 1TB of fs space. With inode64<br>
enabled, the fs can allocate new inode chunks throughout the disk, so<br>
long as there is enough contiguous space. See the following:<br>
<br>
<a href="http://xfs.org/index.php/XFS_FAQ#Q:_What_is_the_inode64_mount_option_for.3F" target="_blank">http://xfs.org/index.php/XFS_FAQ#Q:_What_is_the_inode64_mount_option_for.3F</a><br>
<br>
FWIW, I wouldn't continue using the nobarrier option unless you know<br>
what you're doing and you can safely run with barriers disabled (see the<br>
same FAQ linked above for information on barrier support). It is<br>
completely unrelated to this particular problem.<br>
<span class="HOEnZb"><font color="#888888"><br>
Brian<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> Pat<br>
><br>
><br>
>> On 02/05/2014 03:39 PM, Pat Haley wrote:<br>
>>> Hi Brian,<br>
>>><br>
>>> I tried both just using touch to create<br>
>>> an empty file and copying a small (<1kb)<br>
>>> file. Neither worked.<br>
>>><br>
>>> Note: currently the disk served by gluster-0-1<br>
>>> is mounted as<br>
>>><br>
>>> /dev/sdb1 /mseas-data-0-1 xfs<br>
>>> defaults 1 0<br>
>>><br>
>>> I have received some advice to change the mount<br>
>>> to nobarrier,inode64,delaylog<br>
>>> Would this be compatible with gluster?<br>
>>><br>
>><br>
>> That suggests XFS cannot allocate more inodes. inode64 is worth a try if<br>
>> that is not currently enabled. I don't think it should make a difference<br>
>> for gluster. The other options are unrelated and should have no effect.<br>
>><br>
>> Brian<br>
>><br>
>>> Pat<br>
>>><br>
>>><br>
>>>> On 02/04/2014 02:14 PM, Jeff Darcy wrote:<br>
>>>>>> I tried to "go behind" gluster and directly<br>
>>>>>> write a file to the nfs filesystem on gluster-0-1.<br>
>>>>>><br>
>>>>>> If I try to write to /mseas-data-0-1 (the file<br>
>>>>>> space served by gluster-0-1) directly I still<br>
>>>>>> get the "No space left on device" error.<br>
>>>>>> (df -h still shows 784G on that disk)<br>
>>>>>><br>
>>>> Are you writing to an existing file or attempting to create a new one?<br>
>>>> Can you simply create a new, empty file on your backend (i.e., touch<br>
>>>> mynewfile)?<br>
>>>><br>
>>>> Brian<br>
>>>><br>
>>>>>> If I try to write to the system disk<br>
>>>>>> (the only other area) there is no problem.<br>
>>>>>><br>
>>>>>> I don't have any portion of the disk served by<br>
>>>>>> gluster-0-1 that is not under gluster, so I<br>
>>>>>> can't try to write to a non-gluster portion of<br>
>>>>>> the disk.<br>
>>>>>><br>
>>>>>> Does this suggest anything?<br>
>>>>> Quota limit?<br>
>>>>> _______________________________________________<br>
>>>>> Gluster-users mailing list<br>
>>>>> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
>>>>> <a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
>>>>><br>
>>><br>
>><br>
><br>
><br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Justin Dossey<br>CTO, PodOmatic<div><br></div>
</div>