Improved TianoCore build procedure; can now build both x86-64 and x86

[refind] / BUILDING.txt

From rEFIt to rEFInd
====================

rEFInd is derived from rEFIt (http://refit.sourceforge.net), but the two
programs support different build environments. rEFIt was created with
Intel's EFI Application Toolkit
(http://www.intel.com/technology/efi/toolkit_overview.htm) or TianoCore's
EFI Toolkit (https://efi-toolkit.tianocore.org), along with Microsoft's
Visual C compiler.

Compiling the source code provided on the rEFIt site under Linux never
worked for me, although the documentation claimed it would. Apparently
other Linux developers have run into the same problem; Debian provides a
rEFIt package (http://packages.debian.org/sid/refit) that includes
extensive patches to enable the program to compile under Linux using the
GNU-EFI package (http://sourceforge.net/projects/gnu-efi/). Although
GNU-EFI is less sophisticated than recent versions of TianoCore's toolkit,
GNU-EFI is my preferred environment because it's provided with many Linux
distributions and it was easy to get started with rEFInd development by
using GNU-EFI and the Debian rEFIt package as a starting point.

Over time, though, I've found that the recent TianoCore EDK2 toolkit has
its advantages. Two features, in particular, require the TianoCore EDK2
toolkit:

- The EFI filesystem drivers, added with rEFInd 0.4.0. This requirement is
  a consequence of the derivation of the drivers, which is via VirtualBox
  and the Clover boot loader, both of which are based on EDK2.

- The legacy BIOS boot feature for UEFI-based PCs. EDK2 is needed in this
  case because of features unique to that environment. Note that the legacy
  BIOS boot feature for Macs works when rEFInd is built via either GNU-EFI
  or the TianoCore EDK2.

For these reasons, effective with rEFInd 0.4.6, I've switched the primary
build environment from GNU-EFI to TianoCore EDK2. The rEFInd binary itself
still builds via GNU-EFI, but you must pass the "gnuefi" build target to
make in order to build in this way, and the resulting binary can't boot
BIOS-based OSes on UEFI PCs.

I've dropped ancillary programs, such as the gptsync program, from rEFInd.
You can still use these tools with rEFInd, but you'll need to install them
separately.


Requirements
============

To compile rEFInd, you'll need the following:

* A Linux installation. Note that this installation does NOT need to be
  EFI-based. It can be 32- or 64-bit, but unless you use a cross-compiler
  (which I've not tested), it must be the appropriate bit width for your
  EFI implementation. (Normally that means 64-bit.) If you don't normally
  run Linux, you can run it in a VirtualBox or similar virtual machine. (I
  describe some unsupported non-Linux build options shortly.)

* A standard set of Linux development tools, based on GCC.

* One of the following:

  * The TianoCore EDK2 package
    (http://sourceforge.net/projects/tianocore/). I've tested using the
    UDK2010.SR1 and UDK2010.SR1.UP1 variants
    (http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=UDK2010),
    which are "frozen," rather than the main EDK2 development branch, which
    is changing as the developers add features, fix bugs, and so on. Using
    TianoCore EDK2 is supported in rEFInd version 0.4.3 and later (0.4.0
    and later for the filesystem drivers only). See below for TianoCore
    setup instructions.

  * The GNU-EFI package (http://sourceforge.net/projects/gnu-efi/). You can
    install this from a package called "gnu-efi"; however, rEFInd relies on
    features that were added in (I think) 3.0l to provide driver-loading
    capabilities. The versions I've used and that work are 3.0p and 3.0q.
    As of 5/2012, most Linux distributions seem to deliver rather elderly
    versions of GNU-EFI, so you may need to download the latest source
    code, compile it, and install it locally. Since rEFInd version 0.2.7,
    the Makefiles assume this (see below). The legacy BIOS boot support on
    UEFI-based PCs doesn't work when GNU-EFI is compiled under GNU-EFI, so
    as of rEFInd 0.4.6, GNU-EFI is no longer the primary build environment,
    although it's easier to set up on a Linux system.

It's possible to use a non-Linux platform to compile rEFInd. To the best of
my knowledge, the rEFInd code doesn't rely on anything Linux-specific in
its build requirements, and GNU-EFI's Sourceforge page indicates that it
works under Windows and OS X, too; however, my one attempt to compile
GNU-EFI under OS X failed. I've received one report that rEFInd compiles
successfully with Clang and the TianoCore toolkit under OS X by adding the
refind.inf file to a .dsc file that you use for your own projects. You can
find brief instructions here (note that this is not my documentation):

https://github.com/snarez/refind-edk2

Under Windows, you would need to either create a project or Makefile for
your non-GCC compiler or use a GCC port, such as MinGW
(http://www.mingw.org). You'd probably need to adjust the Makefiles in the
latter case. A procedure similar to that used under OS X might work using
GCC or Microsoft's C compiler, but I haven't tested this.


Preparing Your Development Kit
==============================

If you want to build the rEFInd binary but not the drivers, if you don't
care about booting BIOS-based OSes on UEFI PCs, and if you're using Linux,
GNU-EFI is the easiest way to do the job. I don't describe its setup here
because it's likely to be fairly easy. If your distribution provides a
recent enough version, you should be able to install a package called
gnu-efi and be done with it. If not, you'll need to download the source
code tarball, build it, and install it. This process is fairly typical of
Linux packages. Read the GNU-EFI documentation if you need help. If you're
using GNU-EFI, you can skip the rest of this section.

To build the EFI drivers, or if you want support for booting BIOS-based
OSes on UEFI PCs, the TianoCore toolkit is required. You might also want to
use it if you have problems with GNU-EFI or if you want to build rEFInd on
a non-Linux platform. Unfortunately, the TianoCore toolkit is weird by
Linux programming standards. It's also quite large -- it's intended as a
means to develop a complete EFI firmware implementation, so it contains
much more code than is needed to develop standalone EFI applications. I
don't know of any Linux distribution packages for it in RPM, Debian package
file, or other formats; you MUST install the kit from source code using its
own unusual compilation procedure. The installation documentation also
omits at least one step and is a bit unclear about others. Here's how I
installed the toolkit:

1) Download UDK2010.SR1.UP1 from
   https://sourceforge.net/apps/mediawiki/tianocore/index.php?title=UDK2010.

2) Type "mkdir /usr/local/UDK2010". You can use another directory, but the
   Makefile for rEFInd's EFI drivers assumes this location. You'll need to
   edit the EDK2BASE line in the Make.common file if you install somewhere
   else.

3) Type "cd /usr/local/UDK2010".

3) Unzip the downloaded file (UDK2010.SR1.UP1.Complete.MyWorkSpace.zip) in
   the current directory (/usr/local/UDK2010). This creates a handful of
   files, including a tarball and a couple of .zip files.

4) Type "unzip UDK2010.SR1.UP1.MyWorkSpace.zip". This extracts the
   platform-neutral portion of the development kit.

5) Type "cd MyWorkSpace".

6) Type "tar xvf ../BaseTools\(Unix\).tar". This extracts the
   Linux/Unix-specific portions of the toolkit.

7) Follow the build instructions at
   https://sourceforge.net/apps/mediawiki/tianocore/index.php?title=Using_EDK_II_with_Native_GCC_4.4;
   however, a few changes are required, as detailed below....

8) Type "source edksetup.sh BaseTools". This sets up some environment
   variables, so subsequent steps (NOT including compiling the rEFInd EFI
   drivers) must be typed in the shell you use for this step.

9) Edit Conf/target.txt and change the following:
   - ACTIVE_PLATFORM = MdePkg/MdePkg.dsc
   - TARGET = RELEASE (DEBUG might work, but I've not tested it).
   - TARGET_ARCH = X64 (on x86-64; leave this as IA32 on x86). If you plan
     to build both architectures, you can set this to "IA32 X64".
   - TOOL_CHAIN_TAG = GCC46 (or other value depending on your GCC version;
     type "gcc -v" to learn your GCC version number). Note that GCC 4.7
     doesn't have its own entry, so use GCC46 for GCC 4.7.
   The TianoCore Makefiles read some of these variables from this file
   and use them when accessing directories, so be sure to type these
   entries in the case specified.

10) The documentation refers to editing Conf/tools_def.txt in addition to
    Conf/target.txt, but doesn't specify what to change in
    Conf/tools_def.txt. I haven't found it necessary to make any changes in
    Conf/tools_def.txt EXCEPT when using GCC 4.7 on a Fedora 17 system.
    (I haven't used GCC 4.7 on other platforms, so this may well be
    necessary on other systems, too.) With that setup, I found it
    necessary to change the following line:
    *_GCC46_X64_ASM_FLAGS            = DEF(GCC46_ASM_FLAGS) -m64 -melf_x86_64
    to:
    *_GCC46_X64_ASM_FLAGS            = DEF(GCC46_ASM_FLAGS) -m64

11) Type "make -C /usr/local/UDK2010/MyWorkSpace/BaseTools/Source/C".
    (This step is not documented on the EDK Web page.) Note that this
    requires the g++ compiler and UUID development libraries.
    
10) Type "build" to build the main set of EDK2 files. This process is
    likely to take a few minutes.

If you installed in a location other than the one I've specified, you must
edit the EDK2BASE variable in the Make.tiano and filesystems/Make.tiano
files in the rEFInd source package. Once the toolkit is installed, you can
build the filesystem drivers or rEFInd, as described below.


Compiling rEFInd
================

With your development system set up, you can compile rEFInd as follows:

1) Download and uncompress the rEFInd source code archive. (If you're
   reading this file, you've probably already done this task.)

2) Open a Linux shell prompt

3) Change into the archive's main directory. You should see several files
   including this BUILDING.txt file and several subdirectories such as
   "refind", "libeg", and "include".

4) Type "make gnuefi" to build with GNU-EFI, or either "make" alone or
   "make tiano" to build with TianoCore EDK2. With any luck, rEFInd will
   compile without error, leaving the "refind_ia32.efi" or "refind_x64.efi"
   file, depending on your platform, in the "refind" subdirectory. If you
   want to build IA32 binaries on an x86-64 (X64) system, type "ARCH=ia32
   make". This works only if you're using the TianoCore build kit, and only
   if you set TARGET_ARCH to either "IA32" or "IA32 X64" in target.txt when
   you set up the TianoCore.

5) The default build process does NOT build the filesystem drivers. If you
   want to build them, you must type "make fs" in the main rEFInd source
   directory. The result is filesystem drivers in the filesystems
   subdirectory, and also copies placed in the drivers subdirectory. You
   must install the TianoCore EDK2 to build the drivers.

If rEFInd doesn't compile correctly, you'll need to track down the source
of the problem. Double-check that you've got all the necessary development
tools installed, including GCC, make, and either GNU-EFI or TianoCore EDK2.
You may also need to adjust the Makefile, Make.common file, or Make.tiano
file for your system. (The main Makefile controls the process for both
toolkits, while Make.common holds GNU-EFI options and Make.tiano holds
TianoCore options.) The most likely thing you'll need to change is the path
to the various GNU-EFI include files and libraries. Since rEFInd 0.2.7, the
default Make.common file includes the following definitions:

EFIINC          = /usr/local/include/efi
GNUEFILIB       = /usr/local/lib
EFILIB          = /usr/local/lib
EFICRT0         = /usr/local/lib

If you've installed GNU-EFI from a distribution's package, you may need to
remove "local" from those paths, and perhaps change references to "lib" to
"lib64". As noted earlier, though, as of 5/2012, most distributions provide
out-of-date GNU-EFI implementations that will not work with rEFInd 0.2.7
and later.

If you're using TianoCore's EDK2, as noted earlier, you may need to adjust
the EDK2BASE variable in Make.tiano and filesystems/Make.tiano.

When I tried to compile rEFInd under Ubuntu 12.04 (i386) using GNU-EFI,
even with a locally-compiled GNU-EFI 3.0p or 3.0q, I got errors like this:

main.o: In function `StartLegacy.isra.0':
main.c:(.text+0x8b1): undefined reference to `__stack_chk_fail_local'
lib.o: In function `ScanVolumeBootcode.part.3':
lib.c:(.text+0xf2f): undefined reference to `__stack_chk_fail_local'
lib.o: In function `ScanExtendedPartition.isra.4':

The solution was to recompile GNU-EFI with the -fno-stack-protector GCC
flag. In GNU-EFI, this can be added to the CFLAGS line in Make.defaults.


Installing rEFInd
=================

With rEFInd compiled, you can install it. The easiest way to do this is
with the install.sh script, which works on both Linux and Mac OS X.
Alternatively, you can type "make install" to install using this script.
Note that this installation copies files to the ESP and uses "efibootmgr"
(on Linux) or "bless" (on OS X) to add rEFInd to the firmware's boot loader
list. The docs/refind/installing.html file provides more details on this
script and its use.

If install.sh doesn't work for you or if you prefer to do the job manually,
you may. On a UEFI-based system, you'll want to copy files on the ESP as
follows:

* Create a directory for rEFInd, such as EFI/refind.
* Copy refind/refind_ia32.efi or refind_x64.efi to the ESP's EFI/refind
  directory.
* Copy refind.conf-sample to the EFI/refind directory as refind.conf.
* Copy the icons subdirectory, including all its files, to EFI/refind.

You'll then need to activate rEFInd in your EFI. This can be done with
tools such as "efibootmgr" under Linux or "bless" under OS X. See the
docs/refind/installing.html file for details.


Note to Distribution Maintainers
================================

The install.sh script, and therefore the "install" target in the Makefile,
installs the program directly to the ESP and it modifies the *CURRENT
COMPUTER's* NVRAM. Thus, you should *NOT* use this target as part of the
build process for your binary packages (RPMs, Debian packages, etc.).
(Gentoo could use it in an ebuild, though....) You COULD, however, install
the files to a directory somewhere (/usr/share/refind or whatever) and then
call install.sh as part of the binary package installation process. Placing
the files directly in /boot/efi/EFI/{distname}/refind and then having a
post-install script call efibootmgr is probably the better way to go,
but this assumes that the ESP is mounted at /boot/efi.


Compiling the EFI Filesystem Drivers
====================================

To build all the drivers, you can type "make fs" from the main directory,
which builds the drivers and places copies in both the filesystems and
drivers subdirectories. If you want to build just one driver, you can
change into the "filesystems" directory and type "make {fsname}", where
{fsname} is a filesystem name -- "ext2", "reiserfs", "iso9660", or "hfs".

To install drivers, you can type "make install" in the "filesystems"
directory. This copies all the drivers to the
"/boot/efi/EFI/refind/drivers" directory. Alternatively, you can copy the
files you want manually.

*CAUTION:* Install drivers for your system's architecture *ONLY*.
Installing drivers for the wrong architecture causes some systems to hang
at boot time.

The drivers all rely on filesystem wrapper code created by rEFIt's author,
Christoph Phisterer. Most of the drivers seem to have passed through
Oracle's VirtualBox project (https://www.virtualbox.org) and the Clover
boot loader project (https://sourceforge.net/projects/cloverefiboot/),
which I used as the source for this build.