<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I am experiencing a problem with the MySQL innodb_file_per_table innodb configuration setting on a distributed-replicated Gluster volume.<div><br></div><div>Gluster Version: 3.2.5</div><div>Mysql: 5.1.61</div><div><br></div><div>I am attempting to configure MySQL with the file_per_table option enabled, as this would seem to provide a lot of benefit when running MySQL on Gluster backend storage. Under default MySQL configuration, MySQL's ibdata file would just be one large file. By using the innodb_file_per_table option, MySQL creates a separate table space (separate ibdata file) for every table. I would like to use innodb_file_per_table because this would spread the distribution of IO across more drives on the underlying Gluster volume, not to mention some other benefits. Refer to <a href="http://dev.mysql.com/doc/refman/5.1/en/innodb-multiple-tablespaces.html">http://dev.mysql.com/doc/refman/5.1/en/innodb-multiple-tablespaces.html</a></div><div><br></div><div>The problem is that the innodb_file_per_table option does not seem to work on the Gluster distributed-replicated volume. When I run mysql_install_db this is what I get:</div><div><br></div><div><div>mysql_install_db </div><div>Installing MySQL system tables...</div><div>Installation of system tables failed! Examine the logs in</div><div>/mnt/glusterfs/db/mysql for more information.</div></div><div><br></div><div><br></div><div>Here is what the mysql error log reports after running mysql_install_db:</div><div><br></div><div><div>120331 17:51:17 [ERROR] /usr/sbin/mysqld: Incorrect information in file: './mysql/user.frm'</div><div>ERROR: 1033 Incorrect information in file: './mysql/user.frm'</div><div>120331 17:51:17 [ERROR] Aborting</div><div>120331 17:51:17 [Note] /usr/sbin/mysqld: Shutdown complete</div></div><div><br></div><div><br></div><div><br></div><div>The DEBUG logs from the Gluster client mount log are at <a href="http://pastebin.com/ZFQWHjEq">http://pastebin.com/ZFQWHjEq</a></div><div><br></div><div><br></div><div>The volume where the mysql data directory is stored was created with this command (where variables were substituted appropriately):</div><div>gluster volume create ${VOLNAME} replica 2 transport tcp ${HOST1}:${BRICK1} ${HOST2}:${BRICK1} ${HOST1}:${BRICK2} ${HOST2}:${BRICK2}</div><div><br></div><div>Based on that volume create command, you can see that I have to servers.</div><div><br></div><div><br></div><div>Mysql actually seems to run just fine with the innodb_file_per_table setting enabled if I create the mysql database via mysql_install_db on a local (non-gluster) file system, then copy the mysql data directory into the gluster mount point, and finally start mysql configured to use the Gluster mount point. However, as soon as I attempt to import a mysql dump file, I hit the same problem. It seems that whenever Mysql attempts to create new innodb/ibdata files with this configuration there is what *feels like* a locking issue.</div><div><br></div><div><br></div><div>If I disable the innodb_file_per_table option I have no apparent problems running MySQL on this Gluster volume.</div><div><br></div><div>One interesting to note is that if I shut down the Gluster daemon on one of the servers, MySQL works as expected with the innodb_file_per_table options enabled. In essence, no replication would be occurring on the Gluster volume when I do this.</div><div><br></div><div><br></div><div>Any help is much appreciated. Thank you.</div><div><br></div><div>Jamie</div><div><br></div><div><br></div><div><br></div></body></html>