xen, mysql and tls
when first booting xen most people like me (who like to ignore most ot the manual reading until later) will find a xen warning about the thread local storage libraries and how they access memory in a way that xen makes slow. The warning offers the following advice on how to disable it:
mv /lib/tls /lib/tls.disabled
So, I did this, and rebooted my machine – no more warning about libtls, but mysqld failed to start…
Personally, I prefer to use postgresql, however a lot of web applications use mysql so I need to have it.
One way around the problem is to leave tls in place and cope with the slow emulation the xen will have to perform.
I wouldn’t suggest this to be a satisfactory solution, but it certainly keeps one going whilst hunting for a better solution.
Time to do some reading… oddly enough the xen wiki has just the informationI was looking for, how to build a xen specific glibc, I’ll go into a little more detail.
I found that both with the CentOS 4.5 machine at work and with the fedora 4 machine I have at home that the patch on the above page was not required for the version of glibc I was using, which for the record (downloaded from my preferred mirror) were
- CentOS 4.5
- mirror.centos.org/4.5/os/SRPMS/glibc-2.3.4-2.36.src.rpm
- fedora 4
- download.fedoralegacy.org/legacy/fedora/4/updates/SRPMS/glibc-2.3.6-3.src.rpm
I downloaded the source rpms for these packages because I need to alter them slightly, once downloaded I installed them
rpm -i glibc-2.3.4-2.36.src.rpm
I then moved to the SPECS directory
cd /usr/src/redhat/SPECS
and prepared the package for building (this was only necessary to check if I required the patch)
rpmbuild -bp glibc.spec
If you find rpmbuild complains that you have packages missing that are required, then install them – I had to install some, your list may be different
yum install texinfo libselinux-devel audit-libs-devel libcap-devel
by looking near line 47 of the file in /usr/src/redhat/BUILD/glibc-2.3.6/sysdeps/unix/i386/sysdep.S (the first file mentioned in the patch – [no I'm not sure why the glibc-2.3.4-2.36 rpm uses the directory glibc-2.3.6]) I found code suggesting the patch was not required
# ifndef NO_TLS_DIRECT_SEG_REFS
movl %eax, %gs:C_SYMBOL_NAME(errno@NTPOFF)
# else
movl %gs:0, %ecx
movl %eax, C_SYMBOL_NAME(errno@NTPOFF)(%ecx)
# endif
However, this is not the end of the story – we need to make sure that when we build this package that NO_TLS_DIRECT_SEG_REFS is set, we know it isn’t by default because we receive the xen warning when we boot our system.
Time to edit the glibc.spec file – the spec file is the script that rpmbuild follows when building binary rpms…
In the centos file I located this section
%ifarch i386
BuildFlags="$BuildFlags -mno-tls-direct-seg-refs"
%endif
By commenting out the first and third lines here, we ensure that NO_TLS_DIRECT_SEG_REFS will be set
I couldn’t find any reference to NO_TLS_DIRECT_SEG_REFS in the fedora 4 spec file so I added a new entry after the last ‘%endif’ line talking about build flags for various architectures
BuildFlags="$BuildFlags -mno-tls-direct-seg-refs"
With this done, the next step (and it takes a while) is to build new binary rpm files using the new spec
rpmbuild -bb glibc.spec
With this done, you should see, somewhere near the end of the rpmbuild output, a number of lines informing you what files have been created – for me they are in /usr/src/redhat/RPMS/i386.
So now all that is left to do is to install the new rpms (and then reboot)
rpm -Uhv glibc-2.3.4-2.36.i386.rpm glibc-devel-2.3.4-2.36.i386.rpm glibc-utils-2.3.4-2.36.i386.rpm glibc-common-2.3.4-2.36.i386.rpm
NOTE: I didn’t install all of the rpms generated, just the ones I required.
If you find you are still receiving the xen tls warning on startup, you might find that the statically linked version of ‘nash’ in your initrd is the problem – this file is included in the mkinitrd rpm. You could install the source rpm and rebuild it, or you could just ignore it, afterall a little slow emulation that only happens at boot time isn’t an enormous problem.