<br><br><div class="gmail_quote">On Thu, Mar 14, 2013 at 10:03 AM, Edward Shishkin <span dir="ltr"><<a href="mailto:edward@redhat.com" target="_blank">edward@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Wed, 13 Mar 2013 16:04:14 -0700<br>
Anand Avati <<a href="mailto:anand.avati@gmail.com">anand.avati@gmail.com</a>> wrote:<br>
<br>
> On Wed, Mar 13, 2013 at 3:05 PM, Edward Shishkin <<a href="mailto:edward@redhat.com">edward@redhat.com</a>><br>
> wrote:<br>
><br>
> ><br>
> > Dear all,<br>
> ><br>
> > This requires openssl of version >= 1.0.1c<br>
> ><br>
> > Is it possible to upgrade the openssl on the machines, which<br>
> > perform smoke tests? Any ideas? I can provide the instructions..<br>
> ><br>
> > I hope this feature will be useful and popular.<br>
> ><br>
> > Thanks,<br>
> > Edward.<br>
> ><br>
> ><br>
<br>
<br>
</div>Hi Avati,<br>
<br>
Actually, 1.0.1 (without 'c') will also work, but, I guess, it won't<br>
make us more happy..<br>
<div class="im"><br>
<br>
> Thanks for posting the patches! Can you give a quick summary of what<br>
> 1.0.1c features are being used in the translator?<br>
<br>
<br>
</div>The following features are used (absent in 1.0.0):<br>
<br>
1) XTS: cipher mode for data encryption;<br>
2) GCM: cipher mode for metadata encryption and authentication;<br>
3) CMAC: mode for authentication of non-encrypted part of metadata;<br>
<div class="im"><br>
<br>
> Is it possible to<br>
> re-implement some code to depend on openssl-1.0.0?<br>
<br>
<br>
</div>Perhaps, we can find a replacement for (3) in the 1.0.0 (using HMACs<br>
instead of CMACs). However, absence of (1) and (2) seems to be a real<br>
problem: 1.0.0 doesn't have algorithms suitable for our purposes.<br>
Existing ones do have different issues. For example, the popular mode<br>
CBC doesn't sustain overwrites with the same key, etc.<br>
<br>
We can copypaste the absent code to the crypt translator. However, it<br>
would mean that this code will be beyond the reach of openssl people,<br>
which is not good..<br>
<div class="im"><br>
<br>
> The build system<br>
> is CentOS 6.3 (~= RHEL 6.3) - which is a very popular "base" for<br>
> deploying glusterfs. I suspect depending on 1.0.1c openssl will be<br>
> more than just a "build server dependency" issue.<br>
<br>
<br>
</div>Maybe it makes sense to ifdef the crypt translator and the related<br>
code (depending on the openssl version, or having the respective<br>
header files, say <openssl/cmac.h>)? I can see similar tricks in<br>
the current Gluster code (HAVE_BD_XLATOR and friends)..<br>
<br></blockquote><div><br></div><div>Either ifdef away the build of the entire translator, or copy-paste the missing functionality in 1.0 into contrib/openssl/ and conditionally compile with these if missing in the build environment?</div>
<div><br></div><div>Avati</div><div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks,<br>
Edward.<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
><br>
> Avati<br>
><br>
><br>
><br>
> ><br>
> > On Wed, 13 Mar 2013 17:25:41 -0400 (EDT)<br>
> > John Mark Walker <<a href="mailto:johnmark@redhat.com">johnmark@redhat.com</a>> wrote:<br>
> ><br>
> > > This marks an interesting development in GlusterFS. If you've been<br>
> > > looking for data encryption, you may want to try this patch.<br>
> > ><br>
> > > -JM<br>
> > ><br>
> > ><br>
> > > -------- Original Message --------<br>
> > > Subject: Change in glusterfs[master]: Transparent data<br>
> > > encryption and metadata authentication... From: "Edward Shishkin<br>
> > > (Code Review)" <<a href="mailto:root@dev.gluster.com">root@dev.gluster.com</a>> To:<br>
> > > CC:<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > Edward Shishkin has uploaded a new change for review.<br>
> > ><br>
> > > Change subject: Transparent data encryption and metadata<br>
> > > authentication in the systems with non-trusted<br>
> > > server<br>
> > ......................................................................<br>
> > ><br>
> > > Transparent data encryption and metadata authentication<br>
> > > in the systems with non-trusted server<br>
> > ><br>
> > > This new functionality can be useful in various cloud<br>
> > > technologies. It is implemented via a pair of the following<br>
> > > interacting translators:<br>
> > ><br>
> > > . encryption/crypt, which works on client side and performs<br>
> > > encryption and authentication;<br>
> > > . features/oplock, which works on server side and resolves<br>
> > > "conflicts"<br>
> > ><br>
> > > 1. The class of algorithms for data encryption,<br>
> > > that can be supported by this pair of translators<br>
> > ><br>
> > > The mentioned pair of translators can support any atomic symmetric<br>
> > > block cipher algorithms (which require to pad plain/cipher text<br>
> > > before performing encryption/decryption transform (see glossary<br>
> > > in atom.c for definitions). In particular, it can support<br>
> > > algorithms with the EOF issue (which require to pad the end of<br>
> > > file by extra-data).<br>
> > ><br>
> > > In most cases crypt translator translates the pair (offset, count)<br>
> > > passed by user to different values, and resolves individual<br>
> > > ->write() ->truncate(), etc. file operations to read-modify-write<br>
> > > sequences.<br>
> > ><br>
> > > A volume can contain files encrypted by different algorithms. For<br>
> > > newly created files one can specify desirable algorithm at mount<br>
> > > time via a respective option of crypt translator.<br>
> > ><br>
> > > Currently only one algorithm is supported: AES_XTS.<br>
> > ><br>
> > > Example of algorithms, which can not be supported by this pair of<br>
> > > translators:<br>
> > ><br>
> > > 1. Asymmetric block cipher algorithms, which inflate data, e.g.<br>
> > > RSA; 2. Symmetric block cipher algorithms with inline MACs for<br>
> > > data authentication.<br>
> > ><br>
> > > 2. Implementation notes.<br>
> > ><br>
> > > a) Atomic algorithms<br>
> > ><br>
> > > Since any process in a stackable file system manipulates with<br>
> > > local data (which can be obsoleted by local data of another<br>
> > > process), atomic cipher algorithms without proper support can<br>
> > > lead to non-POSIX behavior. To resolve the "collisions" we<br>
> > > introduce a special helper translator (features/oplock), which<br>
> > > works on the server and manages requests (queues, grants access)<br>
> > > of read/write issued by the clients. When an exclusive access is<br>
> > > granted to client, the last one performs cipher transform and<br>
> > > proceeds the stack. After all the client's work is done on the<br>
> > > server, the oplock translator drops the access, and wakes up the<br>
> > > next request in the queue (if any). Our implementation guarantees<br>
> > > that an access will be granted to every concurrent process, which<br>
> > > accesses the same file (i.e. the process won't hang).<br>
> > ><br>
> > > b) Algorithms with EOF issue<br>
> > ><br>
> > > Such algorithms require to pad the end of file with some<br>
> > > extra-data. Without proper support this will result in losing<br>
> > > information about real file size. Keeping a track of real file<br>
> > > size is a responsibility of the mentioned features/oplock<br>
> > > translator. When writing/truncating a file, the oplock translator<br>
> > > cuts the padding and stores the last one as a special extended<br>
> > > attribute with the key "trusted.ceof". When reading a file,<br>
> > > oplock translator appends the respective padding. So, in the<br>
> > > bricks every file has its real size.<br>
> > ><br>
> > > Comment. This makes transparent encryption incompatible with<br>
> > > GlusterFS striping and replication translators, which spawn<br>
> > > extra-writes to stripe/replica files without due interaction with<br>
> > > the oplock translator.<br>
> > ><br>
> > > 3. Non-trusted servers and<br>
> > > Metadata authentication<br>
> > ><br>
> > > We assume that server, where user's data is stored on is<br>
> > > non-trusted. It means that the server can be subjected to various<br>
> > > attacks directed to reveal user's encrypted personal data. We<br>
> > > provide protection against such attacks.<br>
> > ><br>
> > > Every encrypted file has specific private attributes (cipher<br>
> > > algorithm id, atom size, trusted object id, etc), which are<br>
> > > packed to a string (so-called "format string") and stored as a<br>
> > > special extended attribute with the key "trusted.cfmt". We<br>
> > > protect the string from tampering. This protection is mandatory,<br>
> > > hardcoded and is always on. Without such protection various<br>
> > > attacks (based on extending the scope of per-file secret keys)<br>
> > > are possible.<br>
> > ><br>
> > > Our authentication method has been developed in tight<br>
> > > collaboration with Red Hat security team and is implemented as<br>
> > > "metadata loader of version 1" (see file metadata.c). This method<br>
> > > is NIST-compliant and is based on checking 8-byte per-link MACs<br>
> > > created(updated) by FOP->create(), FOP->link(), FOP->unlink(),<br>
> > > FOP->rename() by the following unique entities:<br>
> > ><br>
> > > . link name;<br>
> > > . trusted file's uuid, specially created on the (trusted) client<br>
> > > side<br>
> > ><br>
> > > Every time, before manipulating with a file, we check it's MACs at<br>
> > > FOP->open() time. Some FOPs don't require a file to be opened<br>
> > > (e.g. FOP->truncate()). In such cases the crypt translator opens<br>
> > > the file mandatory.<br>
> > ><br>
> > > 4. Generating keys<br>
> > ><br>
> > > Unique per-file keys are derived by NIST-compliant methods (file<br>
> > > keys.c) from the<br>
> > ><br>
> > > a) parent key;<br>
> > > b) unique trusted object-id of the file;<br>
> > ><br>
> > > Per-volume master key, provided by user at mount time is in the<br>
> > > root of this "tree of keys".<br>
> > ><br>
> > > Those keys are used to:<br>
> > ><br>
> > > 1) encrypt/decrypt file data;<br>
> > > 2) encrypt/decrypt file metadata;<br>
> > > 3) create per-file and per-link MACs for metadata authentication.<br>
> > ><br>
> > > 5. Instructions<br>
> > > how to use the new functionality<br>
> > ><br>
> > > 1) Specify an option "encrypt" when creating a volume.<br>
> > ><br>
> > > Example:<br>
> > > # gluster volume create myvol encrypt pepelac:/root/exp8<br>
> > ><br>
> > > 2) On the client side, when mounting a volume, specify the<br>
> > > absolute name of the file, which contains per-volume master key,<br>
> > > overriding the option "key" of the crypt translator. This file<br>
> > > should contain 256-bit AES key in the hex form, i.e. 64 symbols.<br>
> > > Crypt translator accepts the first 64 symbols of the specified<br>
> > > file. Other extra-symbols are ignored.<br>
> > > After successful mount the file with master key may be removed.<br>
> > ><br>
> > > Example:<br>
> > > # glusterfs --xlator-option=myvol-crypt.key=/home/edward/mykey<br>
> > > \ --volfile-id=myvol --volfile-server=pepelac /mnt/gluster<br>
> > ><br>
> > > WARNING! Losing the master key means losing the data of the whole<br>
> > > volume without any chances to recovery.<br>
> > ><br>
> > > 6. Options of the crypt translator<br>
> > ><br>
> > > . "key" (specifies names of the file which contains per-volume<br>
> > > master key); . "kbits" (specifies size of per-file key for data<br>
> > > encryption), possible values:<br>
> > > . "256" default value<br>
> > > . "512"<br>
> > > . "blocksize" (specifies the atom size), possible values:<br>
> > > . "512"<br>
> > > . "1024"<br>
> > > . "2048"<br>
> > > . "4096" default value;<br>
> > > . id of algorithm for data encryption (hidden option);<br>
> > > . id of metadata loader (hidden option);<br>
> > ><br>
> > > 7. Test cases<br>
> > ><br>
> > > Any workload, which involves the following file operations:<br>
> > ><br>
> > > ->create();<br>
> > > ->open();<br>
> > > ->readv();<br>
> > > ->writev();<br>
> > > ->truncate();<br>
> > > ->ftruncate();<br>
> > > ->link();<br>
> > > ->unlink();<br>
> > > ->rename();<br>
> > ><br>
> > > 8. TODOs:<br>
> > ><br>
> > > 1) Currently iov_len coincides with atom_size (4K by default). We<br>
> > > can introduce larger units for IOs to improve performance.<br>
> > ><br>
> > > 2) Show encryption status (on/off) of the volume in gluster volume<br>
> > > info.<br>
> > ><br>
> > > Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee<br>
> > > Signed-off-by: Edward Shishkin <<a href="mailto:edward@redhat.com">edward@redhat.com</a>><br>
> > > ---<br>
> > > M cli/src/cli-cmd-parser.c<br>
> > > M <a href="http://configure.ac" target="_blank">configure.ac</a><br>
> > > M doc/gluster.8<br>
> > > M xlators/encryption/Makefile.am<br>
> > > A xlators/encryption/crypt/Makefile.am<br>
> > > A xlators/encryption/crypt/src/Makefile.am<br>
> > > A xlators/encryption/crypt/src/atom.c<br>
> > > A xlators/encryption/crypt/src/crypt-common.h<br>
> > > A xlators/encryption/crypt/src/crypt-mem-types.h<br>
> > > A xlators/encryption/crypt/src/crypt.c<br>
> > > A xlators/encryption/crypt/src/crypt.h<br>
> > > A xlators/encryption/crypt/src/data.c<br>
> > > A xlators/encryption/crypt/src/keys.c<br>
> > > A xlators/encryption/crypt/src/metadata.c<br>
> > > A xlators/encryption/crypt/src/metadata.h<br>
> > > M xlators/features/Makefile.am<br>
> > > A xlators/features/oplock/Makefile.am<br>
> > > A xlators/features/oplock/src/Makefile.am<br>
> > > A xlators/features/oplock/src/oplock-mem-types.h<br>
> > > A xlators/features/oplock/src/oplock.c<br>
> > > A xlators/features/oplock/src/oplock.h<br>
> > > M xlators/mgmt/glusterd/src/glusterd-volgen.c<br>
> > > M xlators/mgmt/glusterd/src/glusterd-volume-ops.c<br>
> > > M xlators/mgmt/glusterd/src/glusterd.h<br>
> > > 24 files changed, 10,159 insertions(+), 19 deletions(-)<br>
> > ><br>
> > ><br>
> > > git pull ssh://<a href="http://git.gluster.org/glusterfs" target="_blank">git.gluster.org/glusterfs</a> refs/changes/67/4667/1<br>
> > > --<br>
> > > To view, visit <a href="http://review.gluster.org/4667" target="_blank">http://review.gluster.org/4667</a><br>
> > > To unsubscribe, visit <a href="http://review.gluster.org/settings" target="_blank">http://review.gluster.org/settings</a><br>
> > ><br>
> > > Gerrit-MessageType: newchange<br>
> > > Gerrit-Change-Id: I2601fe95c5c4dc5b22308a53d0cbdc071d5e5cee<br>
> > > Gerrit-PatchSet: 1<br>
> > > Gerrit-Project: glusterfs<br>
> > > Gerrit-Branch: master<br>
> > > Gerrit-Owner: Edward Shishkin <<a href="mailto:edward.shishkin@gmail.com">edward.shishkin@gmail.com</a>><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Gluster-devel mailing list<br>
> > <a href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
> > <a href="https://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
> ><br>
<br>
</div></div></blockquote></div><br>