How would you handle ->lseek()? Not so much of an issue for Approach 1, but with interleaved HMACs...<br><br><div class="gmail_quote">On Thu, Jul 14, 2011 at 5:01 PM, Edward Shishkin <span dir="ltr"><<a href="mailto:edward@redhat.com">edward@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hello everyone,<br>
any comments, suggestions are welcome..<br>
<br>
<br>
<br>
Handling EOBs (end-of-blocks) for transparent<br>
encryption, checking integrity and data authentication<br>
<br>
DRAFT<br>
<br>
<br>
<br>
This was designed for CloudFS, which uses 2-level protocol (high and<br>
low) supported by xlators which reside on server and client sides<br>
respectively.<br>
<br>
<br>
Definition of EOB. Storage class<br>
<br>
<br>
If file size isn't a multiple of cblock (cipher block) size, then we<br>
also need to store special padding needed to decrypt its last block<br>
with some cipher modes like CBC. This padding contains a part of<br>
ciphertext and must be considered as a part of this file. We'll call<br>
this padding end-of-file (EOF). If plain text has size a multiple of<br>
cblock size, then encrypted file won't have (or will have empty) EOF.<br>
<br>
Signatures (HMACs, etc) for checking integrity, data authentication,<br>
etc. have the same nature as EOF. Every such signature is created<br>
for some logical block in a file. This is not a padding though, as<br>
in the case of EOF, but anyway such signatures are associated with<br>
file's data, and we'll consider a class of object, which includes<br>
EOFs, HMACs, etc, and call them EOBs (end-of-block).<br>
<br>
We define storage class of EOBs as "data", i.e. this can be considered<br>
as part of file's data: we can not read/write data block without<br>
reading/writing its EOB.<br>
<br>
<br>
Storing EOBs. Approaches and Issues<br>
<br>
<br>
Approach 1: Storing EOBs as xattr values.<br>
<br>
<br>
In this case we store a file in parts which are not adjacent<br>
from the standpoint of Cloudfs. That said we need to split<br>
read, and this makes this operation inatomic. This means<br>
that read(2) will return data compound of parts of different<br>
"versions".<br>
<br>
Example:<br>
<br>
Suppose we have a file F stored in 2 different parts F1 and F2.<br>
<br>
Process A writes a file F (to be of version 1);<br>
Process B reads a file F (part F1);<br>
Process C writes a file F (to be of version 2);<br>
Process B reads a file F (part F2);<br>
<br>
As the result process B returns data compound of<br>
parts of different versions 1 and 2.<br>
<br>
This non-atomicity is different from the non-atomicity that takes<br>
place in the kernel (local file systems): kernel guarantees<br>
that all PAGE_SIZE reads with PAGE_SIZE-aligned offsets are<br>
atomic (this is because reads and writes in kernel acquire<br>
page locks). Whereas, in our case we'll have that F2 doesn't<br>
necessarily have PAGE_SIZE-aligned offset.<br>
<br>
That said it can happen that we'll get complaints from users,<br>
who don't expect such non-atomicity. Moreover, in the case when<br>
EOBs are HMACs for checking integrity, or authentication we'll<br>
have false positives, as nobody guarantees that versions of HMAC<br>
and respective data block will coincide.<br>
<br>
Solution:<br>
<br>
In this approach we need to serialize truncates, appending<br>
writes and sequences RbRe (read block, read EOB).<br>
<br>
<br>
Approach 2: Storing in file's body.<br>
<br>
<br>
In this case EOBs are stored in file's body (via appending to<br>
a file in the case of EOF, or interspacing a file with HMACs,<br>
etc). So file with his EOBs is the whole from the standpoint<br>
of Cloudfs, and there is no problems with atomicity specific<br>
to Approach 1.<br>
<br>
However, in this case all our files maintained by low-level<br>
local fs will have increased sizes (added total size of all EOBs).<br>
So that actual file size must be stored as additional attribute<br>
(e.g. as xattr value).<br>
<br>
->open() method of the high-level translator loads actual<br>
file size to the cloudfs-specific part of inode via fetching<br>
->getxattr(), so that it is persistent in the memory on server.<br>
<br>
Any ->truncate() and appending ->write() of the high-level<br>
xlator update in-core and on-disk actual sizes simultaneously<br>
(via fetching ->setxattr() for the last one). This actual size<br>
is what should be returned to user by ->fstat(), ->lookup(),<br>
etc. as st_size.<br>
<br>
______________________________<u></u>_________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/<u></u>mailman/listinfo/gluster-devel</a><br>
</blockquote></div><br>