<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Sorry I missed your numbers on local read tests. Another thing that
happens to me often in my mix network environment is traffic going
through the wrong network interface one way or another. 70-80MB/s
sounds like 1GB/s and not 10GB/s. Do you have a 1GB/s interface too
that read traffics might be going through?<br>
<br>
...<br>
ling<br>
<br>
On 02/28/2013 05:05 PM, Thomas Wakefield wrote:
<blockquote cite="mid:393E92B3-AD63-4DC2-8DD0-54A612617C89@iges.org"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
I get great speed locally, it's only when I add gluster in that it
slows down. I get 2GB/s locally to the exact same brick. It's
gluster that is having the read issue (80MB/s). But Gluster can
write just fine, 800MB/s.
<div><br>
</div>
<div>The blockdev idea is a good one, and I have already done it.
Thanks though.</div>
<div><br>
</div>
<div>-Tom</div>
<div><br>
</div>
<div>
<div>
<div>On Feb 28, 2013, at 7:53 PM, Ling Ho <<a
moz-do-not-send="true"
href="mailto:ling@slac.stanford.edu">ling@slac.stanford.edu</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Tom,<br>
<br>
What type of disks do you have? If they are raid 5 or 6,
have you try setting the read-ahead size to 8192 or 16384
(blockdev --setra 8192 /dev/<sd?> ?<br>
<br>
...<br>
ling<br>
<br>
On 02/28/2013 04:23 PM, Thomas Wakefield wrote:
<blockquote
cite="mid:FFFBE352-E1C4-4FC2-AA5A-6703EC580F9E@iges.org"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
Did anyone else have any ideas on performance tuning for
reads?
<div><br>
<div>
<div>On Feb 27, 2013, at 9:29 PM, Thomas Wakefield
<<a moz-do-not-send="true"
href="mailto:twake@iges.org">twake@iges.org</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<meta http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1">
<div style="word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space; ">Bryan-
<div><br>
</div>
<div>Yes I can write at 700-800MBytes/sec, but i
can only read at <b>70-80 MBytes/sec</b>. I
would be very happy if I could get it to read
at the same speed it can write at. And the
70-80 is sequential, not random for reads,
same exact test commands on the disk server
are in the 2+GB/s range, so I know the disk
server can do it.</div>
<div><br>
</div>
<div>-Tom</div>
<div><br>
</div>
<div><br>
<div>
<div>On Feb 27, 2013, at 7:41 PM, Bryan
Whitehead <<a moz-do-not-send="true"
href="mailto:driver@megahappy.net">driver@megahappy.net</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div dir="ltr">Are your figures 700-800M<b>Byte</b>/sec?
Because that is probably as fast as your
10G nic cards are able to do. You can
test that by trying to push a large
amount of data over nc or ftp.
<div><br>
</div>
<div style=""> Might want to try
Infiniband. 40G cards are pretty
routine.</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Feb 27,
2013 at 3:45 PM, Thomas Wakefield <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:twake@iges.org"
target="_blank">twake@iges.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div style="word-wrap:break-word">I
also get the same performance
running iozone for large file
sizes, iozone -u 1 -r 512k -s 2G
-I -F.
<div> <br>
</div>
<div>Large file IO is what I need
the system to do. I am just
shocked at the huge difference
between local IO and gluster
client IO. I know there should
be some difference, but 10x is
unacceptable.</div>
<div><br>
</div>
<div>-Tom</div>
<div>
<div class="h5">
<div><br>
</div>
<div><br>
</div>
<div><br>
<div>
<div>On Feb 27, 2013, at
5:31 PM, Bryan Whitehead
<<a
moz-do-not-send="true"
href="mailto:driver@megahappy.net" target="_blank">driver@megahappy.net</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">Every
time you open/close a
file or a directory
you will have to wait
for locks which take
time. This is totally
expected.
<div><br>
</div>
<div>Why don't you
share what you want
to do? iozone
benchmarks look like
crap but serving
qcow2 files to qemu
works fantastic for
me. What are you
doing? Make a
benchmark that does
that. If you are
going to have many
files with a
wide variety of
sizes glusterfs/fuse
might not be what
you are looking for.</div>
</div>
<div class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Wed, Feb 27, 2013 at
12:56 PM, Thomas
Wakefield <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:twake@cola.iges.org" target="_blank">twake@cola.iges.org</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0
0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
style="word-wrap:break-word">I
have tested
everything,
small and large
files. I have
used file sizes
ranging from
128k up to
multiple GB
files. All the
reads are bad.
<div> <br>
</div>
<div><br>
<div><br>
</div>
<div>Here is a
fairly
exhaustive
iozone auto
test:</div>
<div><br>
</div>
<div>
<div>
random
random
bkwd record
stride
</div>
<div>
KB reclen
write
rewrite
read
reread
read write
read
rewrite
read fwrite
frewrite
fread freread</div>
<div>
64 4
40222
63492 26868
30060
1620 71037
1572
70570
31294
77096
72475 14736
13928</div>
<div>
64 8
99207
116366
13591
13513 3214
97690
3155 109978
28920
152018
158480 18936
17625</div>
<div>
64 16
230257
253766
25156
28713 10867
223732
8873 244297
54796
303383
312204 15062
13545</div>
<div>
64 32
255943
234481
5735102
7100397
11897 318502
13681
347801
24214
695778
528618 25838
28094</div>
<div>
64 64
214096
681644
6421025
7100397
27453 292156
28117
621657
27338
376062
512471 28569
32534</div>
<div>
128 4
74329
75468 26428
41089
1131 72857
1118
66976
1597 73778
78343
13351 13026</div>
<div>
128 8
100862
135170
24966
16734 2617
118966
2560 120406
39156
125121
146613 16177
16180</div>
<div>
128 16
115114
253983
28212
17854 5307
246180
5431 229843
47335
255920
271173 27256
24445</div>
<div>
128 32
256042
391360
39848
64258 11329
290230
9905 429563
38176
490380
463696 20917
19219</div>
<div>
128 64
248573
592699
4557257
6812590
19583 452366
29263
603357
42967
814915
692017 76327
37604</div>
<div>
128 128
921183
526444
5603747
5379161
45614 390222
65441
826202
41384
662962
1040839
78526 39023</div>
<div>
256 4
76212
77337 40295
32125
1289 71866
1261
64645
1436 57309
53048
23073 29550</div>
<div>
256 8
126922
141976
26237
25130 2566
128058
2565 138981
2985
125060
133603 22840
24955</div>
<div>
256 16
242883
263636
41850
24371 4902
250009
5290 248792
89353
243821
247303 26965
26199</div>
<div>
256 32
409074
439732
40101
39335 11953
436870
11209 430218
83743
409542
479390 30821
27750</div>
<div>
256 64
259935
571502
64840
71847 22537
617161
23383 392047
91852
672010
802614 41673
53111</div>
<div>
256 128
847597
812329
185517
83198 49383
708831
44668 794889
74267
1180188
1662639
54303 41018</div>
<div>
256 256
481324
709299
5217259
5320671
44668 719277
40954
808050
41302
790209
771473 62224
35754</div>
<div>
512 4
77667
75226 35102
29696
1337 66262
1451
67680
1413 69265
69142
42084 27897</div>
<div>
512 8
134311
144341
30144
24646 2102
134143
2209 134699
2296
108110
128616 25104
29123</div>
<div>
512 16
200085
248787
30235
25697 4196
247240
4179 256116
4768
250003
226436 32351
28455</div>
<div>
512 32
330341
439805
26440
39284 8744
457611
8006 424168
125953
425935
448813 27660
26951</div>
<div>
512 64
483906
733729
48747
41121 16032
555938
17424 587256
187343
366977
735740 41700
41548</div>
<div>
512 128
836636
907717
69359
94921 42443
761031
36828 964378
123165
651383
695697 58368
44459</div>
<div>
512 256
520879
860437
145534
135523 40267
847532
31585 663252
69696
1270846
1492545
48822 48092</div>
<div>
512 512
782951
973118
3099691
2942541
42328 871966
46218
911184
49791
953248
1036527
52723 48347</div>
<div>
1024 4
76218
69362 36431
28711
1137 66171
1174
68938
1125 70566
70845
34942 28914</div>
<div>
1024 8
126045
140524
37836
15664 2698
126000
2557 125566
2567
110858
127255 26764
27945</div>
<div>
1024 16
243398
261429
40238
23263 3987
246400
3882 260746
4093
236652
236874 31429
25076</div>
<div>
1024 32
383109
422076
41731
41605 8277
473441
7775 415261
8588
394765
407306 40089
28537</div>
<div>
1024 64
590145
619156
39623
53267 15051
722717
14624 753000
257294
597784
620946 38619
44073</div>
<div>
1024 128
1077836
1124099
56192
64916 36851
1102176
37198 1082454
281548
829175
792604 47975
51913</div>
<div>
1024 256
941918
1074331
72783
81450 26778
1099636
32395 1060013
183218
1024121
995171 44371
45448</div>
<div>
1024 512
697483
1130312
100324
114682 48215
1041758
41480 1058967
90156
994020
1563622
56328 46370</div>
<div>
1024 1024
931702
1087111
4609294
4199201
44191 949834
45594
970656
56674
933525
1075676
44876 46115</div>
<div>
2048 4
71438
67066 58319
38913
1147 44147
1043
42916
967 66416
67205
45953 96750</div>
<div>
2048 8
141926
134567
61101
55445 2596
77528
2564 80402
4258
124211
120747 53888
100337</div>
<div>
2048 16
254344
255585
71550
74500 5410
139365
5201 141484
5171
205521
213113 67048
57304</div>
<div>
2048 32
397833
411261
56676
80027 10440
260034
10126 230238
10814
391665
383379 79333
60877</div>
<div>
2048 64
595167
687205
64262
87327 20772
456430
19960 477064
23190
540220
563096 86812
92565</div>
<div>
2048 128
833585
933403
121926
118621 37700
690020
37575 733254
567449
712337
734006 92011
104934</div>
<div>
2048 256
799003
949499
143688
125659 40871
892757
37977 880494
458281
836263
901375 131332
110237</div>
<div>
2048 512
979936
1040724
120896
138013 54381
859783
48721 780491
279203
1068824
1087085
97886 98078</div>
<div>
2048 1024
901754
987938
53352
53043 72727
1054522
68269 992275
181253
1309480
1524983
121600
95585</div>
<div>
2048 2048
831890
1021540
4257067
3302797
75672 984203
80181
826209
94278
966920
1027159
111832
105921</div>
<div>
4096 4
66195
67316 62171
74785
1328 28963
1329
26397
1223 71470
69317
55903 84915</div>
<div>
4096 8
122221
120057
90537
60958 2598
47312
2468 59783
2640
128674
127872 41285
40422</div>
<div>
4096 16
238321
239251
29336
32121 4153
89262
3986 96930
4608
229970
237108 55039
56983</div>
<div>
4096 32
417110
421356
30974
50000 8382
156676
7886 153841
7900
359585
367288 26611
25952</div>
<div>
4096 64
648008
668066
32193
29389 14830
273265
14822 282211
19653
581898
620798 51281
50218</div>
<div>
4096 128
779422
848564
55594
60253 37108
451296
35908 491361
37567
738163
728059 67681
66440</div>
<div>
4096 256
865623
886986
71368
63947 44255
645961
42689 719491
736707
819696
837641 57059
60347</div>
<div>
4096 512
852099
889650
68870
73891 31185
845224
30259 830153
392334
910442
961983 60083
55558</div>
<div>
4096 1024
710357
867810
29377
29522 49954
846640
43665 926298
213677
986226
1115445
55130 59205</div>
<div>
4096 2048
826479
908420
43191
42075 59684
904022
58601 855664
115105
1418322
1524415
60548 66066</div>
<div>
4096 4096
793351
855111
3232454
3673419
66018 861413
48833
847852
45914
852268
842075 42980
48374</div>
<div>
8192 4
67340
69421 42198
31740
994 23251
1166 16813
837
73827
73126 25169
29610</div>
<div>
8192 8
137150
125622
29131
36439 2051
44342
1988 48930
2315
134183
135367 31080
33573</div>
<div>
8192 16
237366
220826
24810
26584 3576
88004
3769 78717
4289
233751
235355 23302
28742</div>
<div>
8192 32
457447
454404
31594
27750 8141
142022
7846 143984
9322
353147
396188 34203
33265</div>
<div>
8192 64
670645
655259
28630
23255 16669
237476
16965 244968
15607
590365
575320 49998
43305</div>
<div>
8192 128
658676
760982
44197
47802 28693
379523
26614 378328
27184
720997
702038 51707
49733</div>
<div>
8192 256
643370
698683
56233
63165 28846
543952
27745 576739
44014
701007
725534 59611
58985</div>
<div>
8192 512
696884
776793
67258
52705 18711
698854
21004 694124
621695
784812
773331 43101
47659</div>
<div>
8192 1024
729664
810451
15470
15875 31318
801490
38123 812944
301222
804323
832765 54308
53376</div>
<div>
8192 2048
749217
68757 21914
22667
48971 783309
48132
782738
172848
907408
929324 51156
50565</div>
<div>
8192 4096
707677
763960
32063
31928 47809
751692
49560 786339
93445
1046761
1297876
48037 51680</div>
<div>
8192 8192
623817
746288
2815955
3137358
48722 741633
35428
753787
49626
803683
823800 48977
52895</div>
<div>
16384 4
72372
73651 34471
30788
960 23610
903 22316
891
71445
71138 56451
55129</div>
<div>
16384 8
137920
141704
50830
33857 1935
41934
2275 35588
3608
130757
137801 51621
48525</div>
<div>
16384 16
245369
242460
41808
29770 3605
75682
4355 75315
4767
241100
239693 53263
30785</div>
<div>
16384 32
448877
433956
31846
35010 7973
118181
8819 112703
8177
381734
391651 57749
63417</div>
<div>
16384 64
710831
700712
66792
68864 20176
209806
19034 207852
21255
589503
601379 104567
105162</div>
<div>
16384 128
836901
860867
104226
100373 40899
358865
40946 360562
39415
675968
691538 96086
105695</div>
<div>
16384 256
798081
828146
107103
120433 39084
595325
39050 593110
56925
763466
797859 109645
113414</div>
<div>
16384 512
810851
843931
113564
106202 35111
714831
46244 745947
53636
802902
760172 110492
100879</div>
<div>
16384 1024
726399
820219
22106
22987 53087
749053
54781 777705
1075341
772686
809723 100349
96619</div>
<div>
16384 2048
807772
856458
23920
23617 66320
829576
72105 740848
656379
864539
835446 93499
101714</div>
<div>
16384 4096
797470
840596
27270
28132 88784
834938
83428 836809
342898
953274
969699 96499
100657</div>
<div>
16384 8192
760063
812942
41022
40014 102020
812135
104063
820346
229870
1252409
1398306
100959
103793</div>
<div>
16384 16384
743107
825054
3074035
3551062
97740 826175
103597
803926
94681
820474
829106 97168
92018</div>
<div>
32768 64
713528
714585
71310
74296 21135
194177
21551 202998
21110
571030
595232 98598
103373</div>
<div>
32768 128
766698
820615
103028
102269 39013
328409
40325 346403
41084
684604
702513 103887
100689</div>
<div>
32768 256
766591
819138
107557
113878 42088
574878
41778 557989
55045
749922
754434 100964
104866</div>
<div>
32768 512
774365
846915
104255
106094 35280
728957
25829 734316
26990
750614
800955 55013
63750</div>
<div>
32768 1024
759113
806241
22033
22622 37322
761870
45003 766950
54519
755666
741339 70018
66255</div>
<div>
32768 2048
797566
860344
22893
23859 49973
778872
53509 779539
796674
817078
797585 56750
57047</div>
<div>
32768 4096
796693
798985
24267
25222 54410
781810
53053 802845
421421
832858
855811 56748
60507</div>
<div>
32768 8192
741543
795144
29254
29150 58678
830011
55545 833914
221323
914182
952916 56120
60014</div>
<div>
32768 16384
754641
819797
44183
45718 67293
785047
65078 807273
132136
1240414
1267713
56585 56167</div>
<div>
65536 64
663333
656588
32211
31925 17324
202043
15477 215981
15496
592126
597188 86431
93969</div>
<div>
65536 128
716218
791603
92035
95614 40330
330127
39295 346542
39469
697385
714453 105094
98327</div>
<div>
65536 256
769283
798701
110285
110085 38404
535730
39939 585023
52402
741206
749899 98671
100106</div>
<div>
65536 512
728151
814516
112550
108873 32284
690164
38911 708244
50047
778669
765303 98333
92728</div>
<div>
65536 1024
773504
827108
21542
20741 45901
775473
57282 807232
69359
770793
766736 101465
100905</div>
<div>
65536 2048
767661
853446
21077
21260 59825
762711
71985 802488
87290
772402
775685 96832
103189</div>
<div>
65536 4096
767669
828165
20681
21480 79798
829757
91755 838367
1379411
791248
783023 106433
112769</div>
<div>
65536 8192
739291
784946
23100
22454 85945
785134
87081 828489
642088
816781
798275 93307
95318</div>
<div>
65536 16384
736401
791669
26345
26384 93082
799356
91419 751766
385721
911919
884688 109668
96374</div>
<div>
131072 64
658361
661211
79790
81016 19460
182084
19325 195428
19553
551809
554426 96581
81979</div>
<div>
131072 128
759806
838994
51063
56976 33624
326699
36329 360593
35679
710602
716538 81725
51819</div>
<div>
131072 256
787375
843763
54664
65031 33847
557817
35093 581552
45163
750234
758452 82704
83472</div>
<div>
131072 512
777460
818049
87831
87110 27994
705621
31860 723071
38483
779489
793068 88281
86068</div>
<div>
131072 1024
737158
782780
19005
18897 36614
735240
38900 812074
52149
746598
762694 74376
82534</div>
<div>
131072 2048
777386
817330
24169
24693 55171
769662
56082 788848
55937
757716
763147 73508
82351</div>
<div>
131072 4096
751757
792201
25283
25173 48455
813945
56557 837418
59578
749106
757939 67846
70964</div>
<div>
131072 8192
743004
796673
20369
20327 88990
780925
95956 777211
951528
762755
739851 70850
69943</div>
<div>
131072 16384
766023
823611
22333
22142 108337
796852
110848
777704
718954
809861
802822 115896
82903</div>
<div>
262144 64
655769
695392
45973
45668 19083
209417
18768 207272
19133
601329
599267 107705
87261</div>
<div>
262144 128
750052
798999
72711
74713 37329
339480
36854 365587
37187
710898
714372 108400
78806</div>
<div>
262144 256
788355
815575
80897
75141 37080
549762
37521 574393
51731
780429
751131 97395
73842</div>
<div>
262144 512
714994
736649
80224
80195 32986
708196
36048 628508
44043
767699
781863 123267
83558</div>
<div>
262144 1024
793541
800826
20200
20428 45599
752027
48232 798229
57011
715643
763691 76741
73211</div>
<div>
262144 2048
699395
782935
20227
20519 65785
779219
80486 803403
93381
650054
712171 115611
108457</div>
<div>
262144 4096
629531
766261
20544
21563 83107
819917
93008 844103
90625
741879
754931 94114
94475</div>
<div>
262144 8192
731695
804112
21158
21589 82987
787869
84858 799483
95127
731822
737397 95950
95160</div>
<div>
262144 16384
703376
802552
21940
22371 87748
777024
90349 779626
1018679
765278
619969 96051
92734</div>
<div>
524288 64
649937
710342
73778
70893 19734
207179
19367 199709
19493
585910
581340 91902
91443</div>
<div>
524288 128
739270
805153
91507
90395 38547
319049
37988 358516
38123
712602
711814 92463
90851</div>
<div>
524288 256
751279
802496
89481
91022 38869
544097
38244 569142
56130
756367
752466 92677
92875</div>
<div>
524288 512
758393
819788
58085
73437 29030
700120
28767 739236
33925
770177
795879 58963
72689</div>
<div>
524288 1024
778385
818368
18822
18639 38774
798827
50898 799492
55679
801849
800917 92778
82686</div>
<div>
524288 2048
780594
834612
19936
19994 63147
769555
63007 802512
66322
772900
771314 86628
93485</div>
<div>
524288 4096
769806
812570
20165
20137 79896
783615
71217 80835
68550
80166
73733 122559
116844</div>
<div>
524288 8192
797504
887823
123824
124513 94520
844880
86441 710319
65077
636713
819868 87765
80121</div>
<div>
524288 16384
788468
858216
92580
117354 109583
850179
63966 856965
121151
789252
805040 123810
119088</div>
<div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<div><br>
</div>
<div><br>
<div>
<div>On Feb
27, 2013, at
2:46 PM, Bryan
Whitehead <<a
moz-do-not-send="true" href="mailto:driver@megahappy.net"
target="_blank">driver@megahappy.net</a>>
wrote:</div>
<br>
<blockquote
type="cite">
<div dir="ltr">How
are you doing
the read/write
tests on the
fuse/glusterfs
mountpoint?
Many small
files will be
slow because
all the time
is spent
coordinating
locks.</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
On Wed, Feb
27, 2013 at
9:31 AM,
Thomas
Wakefield <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:twake@cola.iges.org" target="_blank">twake@cola.iges.org</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
style="word-wrap:break-word">Help
please-
<div><br>
</div>
<div><br>
</div>
<div>I am
running 3.3.1
on Centos
using a 10GB
network. I
get reasonable
write speeds,
although I
think they
could be
faster. But
my read speeds
are REALLY
slow.</div>
<div><br>
</div>
<div>Executive
summary:</div>
<div><br>
</div>
<div>On
gluster
client-</div>
<div>Writes
average about
700-800MB/s</div>
<div>Reads
average about
70-80MB/s</div>
<div><br>
</div>
<div>On
server-</div>
<div>Writes
average about
1-1.5GB/s</div>
<div>Reads
average about
2-3GB/s</div>
<div><br>
</div>
<div>Any
thoughts?</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Here are
some
additional
details:</div>
<div><br>
</div>
<div>Nothing
interesting in
any of the log
files,
everything is
very quite.</div>
<div>All
servers had no
other load,
and all
clients are
performing the
same way.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>Volume
Name: shared</div>
<div>Type:
Distribute</div>
<div>Volume
ID:
de11cc19-0085-41c3-881e-995cca244620</div>
<div>Status:
Started</div>
<div>Number of
Bricks: 26</div>
<div>Transport-type:
tcp</div>
<div>Bricks:</div>
<div>Brick1:
fs-disk2:/storage/disk2a</div>
<div>Brick2:
fs-disk2:/storage/disk2b</div>
<div>Brick3:
fs-disk2:/storage/disk2d</div>
<div>Brick4:
fs-disk2:/storage/disk2e</div>
<div>Brick5:
fs-disk2:/storage/disk2f</div>
<div>Brick6:
fs-disk2:/storage/disk2g</div>
<div>Brick7:
fs-disk2:/storage/disk2h</div>
<div>Brick8:
fs-disk2:/storage/disk2i</div>
<div> Brick9:
fs-disk2:/storage/disk2j</div>
<div>Brick10:
fs-disk2:/storage/disk2k</div>
<div>Brick11:
fs-disk2:/storage/disk2l</div>
<div>Brick12:
fs-disk2:/storage/disk2m</div>
<div>Brick13:
fs-disk2:/storage/disk2n</div>
<div>Brick14:
fs-disk2:/storage/disk2o</div>
<div> Brick15:
fs-disk2:/storage/disk2p</div>
<div>Brick16:
fs-disk2:/storage/disk2q</div>
<div>Brick17:
fs-disk2:/storage/disk2r</div>
<div>Brick18:
fs-disk2:/storage/disk2s</div>
<div>Brick19:
fs-disk2:/storage/disk2t</div>
<div> Brick20:
fs-disk2:/storage/disk2u</div>
<div>Brick21:
fs-disk2:/storage/disk2v</div>
<div>Brick22:
fs-disk2:/storage/disk2w</div>
<div>Brick23:
fs-disk2:/storage/disk2x</div>
<div>Brick24:
fs-disk3:/storage/disk3a</div>
<div>Brick25:
fs-disk3:/storage/disk3b</div>
<div> Brick26:
fs-disk3:/storage/disk3c</div>
<div>Options
Reconfigured:</div>
<div>performance.write-behind:
on</div>
<div>performance.read-ahead:
on</div>
<div>performance.io-cache:
on</div>
<div>performance.stat-prefetch:
on</div>
<div>performance.quick-read:
on</div>
<div>cluster.min-free-disk:
500GB</div>
<div>nfs.disable:
off</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>sysctl.conf
settings for
10GBe</div>
<div>
<div>#
increase TCP
max buffer
size settable
using
setsockopt()</div>
<div>net.core.rmem_max
= 67108864 </div>
<div>net.core.wmem_max
= 67108864 </div>
<div>#
increase Linux
autotuning TCP
buffer limit</div>
<div>net.ipv4.tcp_rmem
= 4096 87380
67108864</div>
<div>net.ipv4.tcp_wmem
= 4096 65536
67108864</div>
<div>#
increase the
length of the
processor
input queue</div>
<div>net.core.netdev_max_backlog
= 250000</div>
<div>#
recommended
default
congestion
control is
htcp </div>
<div>net.ipv4.tcp_congestion_control=htcp</div>
<div> #
recommended
for hosts with
jumbo frames
enabled</div>
<div>net.ipv4.tcp_mtu_probing=1</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<br>
<br>
<div> <span
style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px">
<div> <span
style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><span
style="text-indent:0px;letter-spacing:normal;font-variant:normal;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:12px;white-space:normal;font-family:Helvetica;word-spacing:0px">
<div
style="word-wrap:break-word">
<div>Thomas W.<br>
Sr. Systems
Administrator
COLA/IGES<br>
<a
moz-do-not-send="true"
href="mailto:twake@cola.iges.org" target="_blank">twake@cola.iges.org</a><br>
</div>
</div>
</span></span></div>
Affiliate <span
style="font-size:12px">Computer Scientist GMU</span></span> </div>
<br>
</div>
<br>
_______________________________________________<br>
Gluster-users
mailing list<br>
<a
moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a
moz-do-not-send="true"
href="http://supercolony.gluster.org/mailman/listinfo/gluster-users"
target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
_______________________________________________<br>
Gluster-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>