<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<tt>Hi,<br>
<br>
<br>
Today we discussed about GlusterFS backup API, our plan is to
provide a tool/api to get list of changed files(Full/incremental)</tt><tt><br>
</tt><tt><br>
</tt><tt>Participants: Me, Kotresh, Ajeet, Shilpa</tt><tt><br>
</tt><tt><br>
</tt><tt>Thanks to Paul Cuzner for providing inputs about pre and
post hooks available in backup utilities like NetBackup.</tt><tt><br>
</tt><tt><br>
</tt><b><tt><br>
</tt></b><b><tt>Initial draft:</tt></b><tt><br>
</tt><tt>==============</tt><tt><br>
</tt><tt><br>
</tt><tt>Case 1 - Registered Consumer</tt><tt><br>
</tt><tt>----------------------------</tt><tt><br>
</tt><tt><br>
</tt><tt>Consumer application has to register by giving a session
name.</tt><tt><br>
</tt><tt><br>
</tt><tt>glusterbackupapi register <sessionname> <host>
<volume></tt><tt><br>
</tt><tt><br>
</tt><tt><br>
<br>
</tt><tt>When the following command run for the first time, it will
do full scan. next onwards it does incremental. Start time for
incremental is last backup time, endtime will be current time.</tt><tt><br>
</tt><tt><br>
</tt><tt>glusterbackupapi <sessionname> --out-file=out.txt</tt><tt><br>
</tt><tt><br>
</tt><tt>--out-file is optional argument, default output file name
is `output.txt`. Output file will have file paths.</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
<br>
</tt><tt>Case 2 - Unregistered Consumer</tt><tt><br>
</tt><tt>-----------------------------</tt><tt><br>
</tt><tt><br>
</tt><tt>Start time and end time information will not be remembered,
every time consumer has to send start time and end time if
incremental.</tt><tt><br>
</tt><tt><br>
</tt><tt>For Full backup,</tt><tt><br>
</tt><tt><br>
</tt><tt> glusterbackupapi full <host> <volume>
--out-file=out.txt</tt><tt><br>
</tt><tt><br>
</tt><tt>For Incremental backup,</tt><tt><br>
</tt><tt><br>
</tt><tt> glusterbackupapi inc <host> <volume>
<STARTTIME> <ENDTIME> --out-file=out.txt</tt><tt><br>
</tt><tt><br>
</tt><tt>where STARTTIME and ENDTIME are in unix timestamp format.</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><b><tt>Technical overview</tt></b><tt><br>
</tt><tt>==================</tt><tt><br>
</tt><tt>1. Using host and volume name arguments, it fetches volume
info and volume status to get the list of up bricks/nodes.</tt><tt><br>
</tt><tt>2. Executes brick/node agent to get required details from
brick. (TBD: communication via RPC/SSH/gluster system:: execute)</tt><tt><br>
</tt><tt>3. If full scan, brick/node agent will gets list of files
from that brick backend and generates output file.</tt><tt><br>
</tt><tt>4. If incremental, it calls Changelog History API, gets
distinct GFID's list and then converts each GFID to path.</tt><tt><br>
</tt><tt>5. Generated output files from each brick node will be
copied to initiator node.</tt><tt><br>
</tt><tt>6. Merges all the output files from bricks and removes
duplicates.</tt><tt><br>
</tt><tt>7. In case of session based access, session information
will be saved by each brick/node agent.</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><b><tt>Issues/Challenges</tt></b><tt><br>
</tt><tt>=================</tt><tt><br>
</tt><tt>1. If timestamp different in gluster nodes. We are
assuming, in a cluster TS will remain same.</tt><tt><br>
</tt><tt>2. If a brick is down, how to handle? We are assuming, all
the bricks should be up to initiate backup(atleast one from each
replica)</tt><tt><br>
</tt><tt>3. If changelog not available, or broken in between start
time and end time, then how to get the incremental files list. As
a prerequisite, changelog should be enabled before backup.</tt><tt><br>
</tt><tt>4. GFID to path conversion, using `find -samefile` or using
`glusterfs.pathinfo` xattr on aux-gfid-mount.</tt><tt><br>
</tt><tt>5. Deleted files, if we get GFID of a deleted file from
changelog how to find path. Do backup api requires deleted files
list?</tt><tt><br>
</tt><tt>6. Storing session info in each brick nodes.</tt><tt><br>
</tt><tt>7. Communication channel between nodes, RPC/SSH/gluster
system:: execute... etc?</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt> Kotresh, Ajeet, Please add if I missed any points.</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt> --</tt><tt><br>
</tt><tt> regards</tt><tt><br>
</tt><tt> Aravinda</tt>
</body>
</html>