<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ks_c_5601-1987">
<style>
<!--
@font-face
{font-family:±¼¸²}
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:"¸¼Àº °íµñ"}
@font-face
{font-family:"\@¸¼Àº °íµñ"}
@font-face
{font-family:Tahoma}
@font-face
{font-family:"\@±¼¸²"}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
p
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{margin:0cm;
margin-bottom:.0001pt;
font-size:9.0pt;
font-family:"¸¼Àº °íµñ"}
span.EmailStyle18
{font-family:"¸¼Àº °íµñ";
color:#1F497D}
span.Char
{font-family:"¸¼Àº °íµñ"}
@page Section1
{margin:3.0cm 72.0pt 72.0pt 72.0pt}
-->
</style><style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1" style="word-wrap: break-word;" lang="KO" link="blue" vlink="purple">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">Hi,<br>
<br>
Did you test GlusterFS write performance (using 'dd') *only* from the client mount ?
<br>
<br>
I ask this because GlusterFS Hadoop plugin does a FUSE mount on *every* node in the cluster. So during the map phase, when jobs get assigned to slaves; all I/O will be done via FUSE (which is mostly reads). Similarly, during Reduce phase, the reduce jobs would
be writing to the FUSE mount (on their respective nodes).<br>
<br>
Can you try doing the 'dd' test on all nodes in the cluster parallely (on the FUSE mount) on the 2x2 Distribute-Replicate setup and let us know the numbers (throughput numbers from all nodes would be helpful, if possible).<br>
<br>
Write performance in HDFS is exceptionally well because of it's aggressive client side caching (HDFS relaxes a POSIX requirement to get higher write throughput).<br>
<br>
Thanks,<br>
-Venky<br>
<br>
<div style="font-family: Times New Roman; color: rgb(0, 0, 0); font-size: 16px;">
<hr tabindex="-1">
<div style="direction: ltr;" id="divRpF220790"><font color="#000000" face="Tahoma" size="2"><b>From:</b> °ø¿ëÁØ(yongjoon kong)/Cloud Computing ±â¼ú´ã´ç/SKCC [andrew.kong@sk.com]<br>
<b>Sent:</b> Wednesday, October 19, 2011 11:04 PM<br>
<b>To:</b> Venky Shankar; andrew; gluster-users@gluster.org<br>
<b>Subject:</b> RE: [Gluster-users] gluster map/reduce performance..<br>
</font><br>
</div>
<div></div>
<div>
<div class="Section1">
<div>
<table class="MsoNormalTable" border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="padding: 0cm;">
<div>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">Yes, I used the GlusterFS plugin.
</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">Gluster version is - 3.3 beta 2.
</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">For the Volumes
</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> Distributed-mirroring volume: Using 4 server and 2(brick)x2(replica) configuration</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> Stripe-mirroring volume : Using 4 Server and 4(stripe count) x 2 (repica) configuration</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">For the Map/reduce system I user 6 server ( 4 is the brick server and other 2 is for just map/reduce )</span></p>
</div>
</td>
</tr>
<tr>
<td style="padding: 3.75pt 3.75pt 0cm 11.25pt;"></td>
</tr>
<tr>
<td style="padding: 3.75pt 3.75pt 0cm 11.25pt;"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">I checked your source file, but I can¡¯t find any clue for the Performance degradation in Merging Stage. ( I think it
is connected with writing) </span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">Actaully, In writing test, Gluster was quite good. So I¡¯m little confused right now.
</span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">Regards</span></p>
<p class="MsoNormal" style="text-align: justify;"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US">Andrew</span></p>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "¸¼Àº °íµñ"; color: rgb(31, 73, 125);" lang="EN-US"> </span></p>
<div>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; -moz-border-top-colors: none; -moz-border-right-colors: none; -moz-border-bottom-colors: none; -moz-border-left-colors: none; -moz-border-image: none; padding: 3pt 0cm 0cm;">
<p class="MsoNormal"><b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif";" lang="EN-US">From:</span></b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif";" lang="EN-US"> gluster-users-bounces@gluster.org [mailto:gluster-users-bounces@gluster.org]
<b>On Behalf Of </b>Venky Shankar<br>
<b>Sent:</b> Thursday, October 20, 2011 1:35 AM<br>
<b>To:</b> andrew; gluster-users@gluster.org<br>
<b>Subject:</b> Re: [Gluster-users] gluster map/reduce performance..</span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<div>
<p class="MsoNormal"><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;" lang="EN-US">Hi there,<br>
<br>
Appreciate if you could share the following info with us:<br>
<br>
* Are you using GlusterFS hadoop plugin (which is here <a href="http://download.gluster.com/pub/gluster/glusterfs/qa-releases/3.3-beta-2/glusterfs-hadoop-0.20.2-0.1.x86_64.rpm" target="_blank">
http://download.gluster.com/pub/gluster/glusterfs/qa-releases/3.3-beta-2/glusterfs-hadoop-0.20.2-0.1.x86_64.rpm</a> and is still in beta) or are you using GlusterFS as an additional layer below Hadoop's FileSystem (HDFS) ?<br>
<br>
The latter is basically configuring Hadoop to use GlusterFS mount point (e.g. FUSE mount) as the data directory for Hadoop's DFS.<br>
<br>
Let us know your setup (including GlusterFS version) to debug further.<br>
<br>
Thanks,<br>
-Venky</span></p>
<div>
<div class="MsoNormal" style="text-align: center;" align="center"><span style="color: black;" lang="EN-US">
<hr align="center" size="2" width="100%">
</span></div>
<div id="divRpF789062">
<p class="MsoNormal" style="margin-bottom: 12pt;"><b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;" lang="EN-US">From:</span></b><span style="font-size: 10pt; font-family: "Tahoma","sans-serif"; color: black;" lang="EN-US">
gluster-users-bounces@gluster.org [gluster-users-bounces@gluster.org] on behalf of andrew [sstrato.kong@gmail.com]<br>
<b>Sent:</b> Wednesday, October 19, 2011 6:15 PM<br>
<b>To:</b> gluster-users@gluster.org<br>
<b>Subject:</b> [Gluster-users] gluster map/reduce performance..</span><span style="color: black;" lang="EN-US"></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">Hi, all, </span></p>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">i try to check the performance of Map/Reduce of Gluster File system. </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">Mapper side speed is quite good and it is sometimes faster than hadoop's map job. </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">But in the Reduce Side job is much slower than hadoop. </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">i analyze the result and i found the primary reason of slow speed is bad performance in Merging stage. </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">Would you have any suggestion for this issue</span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">FYI check the blog <a href="http://storage4com.blogspot.com/" target="_blank">http://storage4com.blogspot.com/</a></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US"> </span></p>
</div>
<div>
<p class="MsoNormal"><span style="color: black;" lang="EN-US">thanks.</span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>