Comprehensive Debian Wheezy Xen
Tutorial
Table of Contents
Compiling a Custom Linux Kernel
The following guide is comprised of my personal documentation and excessive
filtering for human consumption. It details the series of steps I took to successfully
install and compile a custom Linux Kernel, Xen 4.2 unstable, and run three virtual
machines to perform unique tasks all on a single physical computer.
To achieve the desired functionality, I used IOMMU for PCI Passthrough with a
multimedia operating system, granting complete access to a graphics card for GPU
Acceleration.
I am a college student, not a trained professional, and I am sharing this documentation
for educational purposes. Blind use of this document for a production environment in a
business setting would be ill-advised.
To quickly summarize my experiences, I decided to try Xen and began researching in
early January 2012.
I purchased equipment in March 2012, and began what I thought would take "at most
two weeks".
By late April I finally had a (mostly) functional system.
I have been fine tuning the system and my process since then to produce this guide for
others.
The purpose of this guide is to turn a 6 month project into a series of steps that can be
reproduced inside the time frame of 1-2 days.
I wanted a single physical computer, that could handle three computers worth of
separated activities at all times, including these four specifics:
Router/Firewall (PFSense)
DNS & Web Development (Debian Squeeze)
Application Development (Debian Wheezy)
Multimedia & Gaming (Windows 7)
I investigated alternative software, including VMWare's ESXi and Citrix's XenServer.
I had come from a VMWare platform having used both VMware Server 2 and VMWare
Workstation 8 for the same key objectives previously but with a Windows Host, subject
to Windows Updates which let to my investigation.
ESXi was easy to install, but missing numerous drivers for hardware components. I
quickly ruled it out as I wanted a home-use system, and there was no convenient or
well-documented method to installing or even launching a virtual machine from their
on-server interface.
Citrix's XenServer was easily my favorite of the options, it's design made for a great
user experience. Everything was easy to find and do. However it came with a
minimum price tag of $1000, and if I wanted PCI passthrough for graphics cards
$2500. This was beyond my reach as a college student.
I chose Xen since it had support for my hardware, was a free open-source project, and
had a great community with lots of activity.
Things you Will Need:
Compatible Hardware
ATI Graphics Card
Motherboard with UEFI & VT-d & Onboard Graphics
Latest Ubuntu Live DVD
Debian Wheezy Beta1 (Or Newer) Installer
nVidia Cards can be made to work, with extensive patching, in Windows XP and
supposedly Windows 8 Preview, for more information visit
If you plan to pass your graphics card to a virtual machine, you will need either a
second graphics card, onboard graphics, or a second computer to manage your Dom0
system and install virtual machines over VNC.
UEFI compatible boot DVD will save you an undocumented step for setting up a UEFI
boot loader.
Xen is picky, different hardware may yield different results, both at compile time and
runtime. If you want to save yourself some hassle, here is my hardware list and some
suggestions to avoid:
Motherboard:
ASRock Z68 Extreme4 Gen3
CPU:
Intel Core i2600
RAM:
12GB 1333Mhz Corsair XMS (2x2G + 2x4G)
Boot Disk:
240GB OCZ Vertex 3
GPU:
ATI Radeon HD 6870
LAN:
Onboard Broadcom BCM57781
PCIe EXPI9301CTBLK
Hardware and Configurations to Avoid:
NF200 Chipsets are not IOMMU compatible
nVidia Graphics Cards
RAID5 yields horrible performance
NF200 is a PCI Switch for motherboards sporting SLI and CrossFire, avoid it if you
want be able to use those PCI slots for passthrough.
These instructions are for an Intel CPU and ASRock UEFI Motherboard, and may vary
depending on your manufacturer as well as your choice of CPU. If you are unfamiliar
with motherboard configuration, you may want to watch the video for a visual walk
through.
Before starting reset your CMOS so you have a clean slate to work from.
List of key settings:
Turn Legacy USB 3.0 off
Turn on VT-x and VT-d
Set drives to ACHI Mode
Change default video to Onboard
Justification:
Consider yourself warned, if you leave Legacy USB on and leave a backup USB drive
connected, your system will fail to boot. To my surprise that is because it is trying to
boot from that USB drive (even if that drive is USB 2.0), which threw me off so I
recommend disabling that feature.
Most motherboards will have Intel Virtualization enabled by default (VT-x), but that is
not the same as IOMMU, be sure to look for VT-d or check the manual for details. For
my system VT-x was in the CPU configuration area, and VT-d was in the Northbridge
configuration area.
ACHI is a superior choice for performance with modern hard drives, and while RAID is
an alternative that uses ACHI as its underlying type, it increases boot time by checking
RAID configurations, and most onboard RAID is software RAID that is built for
Windows and rarely helpful for Linux.
ASRock boards will use a PCI GPU by default if it is plugged into a PCI slot,
regardless of if any video cables are connected, and if you plan to pass that device to
a virtual machine, you do not want to use it for video.
Select Advanced Options from the menu, and then Advanced Install.
To expedite this process we can select defaults for most options (such as for
networking).
Once we reach Hard Drive Partitioning options, select Manual at the bottom.
Select the disk(s) we will be using by title not partition space and hit enter, we can now
select gpt from the partition tables instead of msdos.
We will then create three partitions:
256MB FAT32 Partition mounted to /boot/efi
256MB Ext4 Partition mounted to /boot
Remainder LVM Partition
Select Configure the Logical Volume Manager and create a volume group.
Create three Logical Volumes:
8GB or More for Linux
20B or More for Home
2GB or More for Swap
Complete LVM Configuration, and then back to our partition settings we can now
select formats and mount points:
8GB LV Ext4 for root (/)
20GB LV Ext4 for /home
2GB LV swap
Optionally add noatime flags to all the Ext4 partitioned disks.
Complete the Partition steps by selecting done and create the partitions.
Continue through the options with defaults, until we get to the main Packages.
De-select Debian Desktop Environment and continue
Do not install a boot loader, select the Continue without a bootloader option and be
sure to jott down the information it provides us for later.
Complete the installation and restart our computer.
Insert the Ubuntu Live DVD and press whatever key opens the boot options menu at
startup.
Select the UEFI boot option from that menu.
When presented with an options screen, press c on the keyboard to access the
Emergency Grub Menu.
Emergency Grub allows us to manually boot our Linux, since we did not install a boot
loader.
Type ls to list the disks, we should have an (hd0) and three gpt partitions.
Assuming the same partitioning as mine, in the below example tab indicates hit the tab
key to auto-complete, and we can either use the data provided when we selected
Continue without a bootloader or just substitute xen for the name we gave our
Volume Group name, and linux for our root (/)Logical Volume:
grub> set root=(hd0,gpt2)
grub> linux /vmlinu_tab_ ro root=/dev/mapper/xen-linux
grub> initrd /initrd_tab_
grub> boot
Gibberish may be displayed when tabbing to auto-complete, this is normal and can be
ignored. After entering boot it will begin starting Linux.
If done successfully a terminal login will be presented, which concludes this segment
of the guide.
In the next segment we will start by enter our credentials and we can start installing
some key packages to get UEFI working.
References
Note:
I have done my best to separate my crazier desires from the average persons needs.
To do this I created bold headers with line breaks for special sub-sections, whose
subsequent contents you can skip if said bold text does not apply to your situation.
Assuming you have just installed Debian Wheezy and are at the login prompt, you
want to login as root so you can install packages (alternatively you can use
su
to
change users if you prefer).
UEFI STEPS
Assuming you came from my previous guide and have just manually booted, your first
objective should be getting an efi bootloader installed:
aptitude install grub-efi-amd64
Once installed you will want to run the install and update scripts:
grub-install
update-grub
END UEFI STEPS
Now let's install a couple more basic packages:
aptitude install ssh sudo
With ssh installed we can, optionlly, access remotely for the remainder of the guide.
With sudo we can add our user to the sudoers group like so (cdelorme is my
username):
usermod -aG sudo cdelorme
If you omit the
a
you will replace all existing groups your user belongs to, so be
careful.
I recommend rebooting now to get ssh working, and also so you can login as your user
to use the sudo command (if you used
su
you may need to logout and log back in for
sudo
to work).
Next we want to prime our server with some general utility packages:
parted
samba
ntp
screen
p7zip-full
Delicious Copy Pasta:
sudo aptitude install parted samba ntp screen p7zip-full
Note: I used to install
ntfsprogs
but the package is currently bugged and eats
CPU cycles, so I would avoid it if possible.
Configuration of samba and ntp will vary by system, I recommend using other guides
online for this since as long as they work they will be fine when we start using Xen. If
you want my configurations specifically, ask and I may consider adding them to a
future edition of this guide.
Development Environment Steps
Since I intend to use Debian Wheezy as a development environment, I also installed
these packages:
openjdk-7-jdk
ant
g++
libboost-dev
gnustep
python
python-dev
Delicious Copy Pasta:
End Development Environment Steps
GUI Environment Steps
To be honest, I don't care for the CD Installation of Gnome3, it's less work but the cost
is 30 minutes of installation time, and a whole slew of packages I never use, so I went
minimalist and found exactly what I needed:
gnome-session
gnome-terminal
gnome-screensaver
xserver-xorg-core
gdm3
gksu
ia32-libs
ia32-libs-gtk
libc6-dev-i386
binfmt-support
Delicious Copy Pasta:
sudo aptitude install openjdk-7-jdk ant g++ libboost-dev gnustep python python-dev
sudo aptitude install gnome-session gnome-terminal gnome-screensaver xserver-xorg-core gdm3 gksu ia32-libs ia32-libs-gtk libc6-dev-i386 binfmt-support
If you intend to use Xen with SDL later on, I recommend replacing
xserver-xorg-core
with
xorg
and
xorg-dev
packages, as these are what we will install later, and they
contain the
xserver-xorg-core
.
As a bare-bones install of Gnome3, it should take roughly 5 minutes as opposed to 30.
At this stage you can run
startx
to launch the GUI environment, but I generally reboot
to test the login screen.
End GUI Environment Steps
Optional GUI Additions
Optional additions include enabling audio, installing some basic multimedia, utilities,
and important software like Guake terminal, Google Chrome, and Sublime Text.
Starting with Audio:
sudo aptitude install alsa-base
sudo alsactl init
Now with GUI utilities:
python-xdg
font-manager
gnome-disk-utility
gnome-screenshot
guake
Delicious Copy Pasta:
Note: Guake has a bug with Gnome3 that prevents it from starting at boot, here is
how you can fix it:
This is actually a very simple edit, in this specific version of guake (located at
/usr/bin/guake
) line 653 contains a
notification.show()
, we just have to wrap it in
a simple try-catch so it doesn't crash when it is run before the GUI is ready to accept a
pynotify event:
try:
notification.show()
except Exception:
pass
Next we want to set Guake to start at login. You can make an auto-start file via:
For google chrome we need to add a new debian source to
/etc/apt/sources.list
,
and download their package key:
sudo aptitude install python-xdg font-manager gnome-disk-utility gnome-screenshot guake
mkdir ~/.config/autostart
cd ~/.config/autostart
echo "[Desktop Entry]" >> guake.desktop
echo "Name=Guake Terminal" >> guake.desktop
echo "Name[pt]=Guake Terminal" >> guake.desktop
echo "Name[pt_BR]=Guake Terminal" >> guake.desktop
echo "Comment=Use the command line in a Quake-like terminal" >> guake.desktop
echo "TryExec=guake" >> guake.desktop
echo "Exec=guake" >> guake.desktop
echo "Icon=/usr/share/pixmaps/guake/guake.png" >> guake.desktop
echo "Type=Application" >> guake.desktop
echo "Categories=GNOME;GTK;Utility;TerminalEmulator;" >> guake.desktop
Next we can install these packages:
google-chrome-unstable
eog
gnash
vlc
gtk-recordmydesktop
Delicious Copy Pasta:
Note: I am aware that google source is for Squeeze, I have had no problems running it
in Wheezy.
After google-chrome has been installed, it will create its own sources.list file entry, so
we want to go back in and remove the line we added from
/etc/apt/sources.list
,
and re-run
sudo aptitude update
to make sure it works.
Finally, I am in love with Sublime Text 2, so I went to their website for the download,
and went through these steps for a complete installation.
Once downloaded, let's extract it and move it to
/usr/local
, and create a new
command to run it from terminal:
su root
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
aptitude update
exit
sudo aptitude install google-chrome-unstable eog gnash vlc gtk-recordmydesktop
tar xf sublime-text.tar.bz2
sudo mv sublime-text /usr/local/
cd /usr/local/sublime-text
sudo ln -s /usr/local/sublime-text/sublime_text /usr/bin/subl
Next let's create a desktop file so we can run it as an application and set default
mimetypes from the GUI. Let's navigate to
/usr/share/applications/
and create a
new file
subl.desktop
with these contents:
[Desktop Entry]
Name=Sublime Text 2
Comment=The Best Text Editor in the World!
TryExec=subl
Exec=subl
Icon=/usr/local/sublime-text/Icon/256x256/sublime_text.png
Type=Application
Next to set the mimetypes we add these lines to
/usr/share/applications/defaults.list
:
text/plain=subl.desktop
text/css=subl.desktop
text/htm=subl.desktop
text/javascript=subl.desktop
text/x-c=subl.desktop
text/csv=subl.desktop
text/x-java-source=subl.desktop
text/java=subl.desktop
There are user-account alternatives that may seem more appropriate, but I wanted
these to apply system-wide.
End Optional GUI Additions
Thus concludes our Debian Wheezy configuration, and we are now ready to move
onto compiling a Custom Linux Kernel.
References
Missing ia32-libs-gtk for Gnome3
Compiling a Custom Linux Kernel
Compiling a custom Linux Kernel will make PCI passthrough much easier, but it also
allows us to improve performance in various areas for Dom0.
For starters let's create a folder for compiling our packages in our home directory:
mkdir -p ~/src/linux
cd ~/src/linux
Next we will install the required packages:
wget
tar
bzip2
build-essential
libncurses-dev
kernel-package
fakeroot
Delicious copy pasta:
To improve compiling speed we can set the concurrency to double the number of
physical CPU cores by running these commands:
su root
echo "CONCURRENCY_LEVEL=8" >> /etc/kernel-pkg.conf
exit
IMPORTANT NOTE: Kernel 3.5 and 3.5.2 are bugged and passthrough with USB
Controllers fails (though VGA Passthrough works for some odd reason). That is why
this guide is using 3.4.9, however 3.3.8 has also been tested and works.
Our next step is downloading, extracting, and inserting the current kernel configuration
before adjusting the kernel we are about to compile:
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.4.9.tar.bz2
tar jxf linux-3.4.9.tar.bz2 && rm linux-3.4.9.tar.bz2
cd linux-3.4.9/
cp /boot/config-3.2.0-3-amd64 .config
make menuconfig
Configuring a kernel is not as complex as it seems, the menu is just a hierarchy of
sudo aptitude install wget tar bzip2 build-essential libncurses-dev kernel-package fakeroot
options, and thankfully you can search for settings with
/
and get details on individual
settings with
h
.
Here is a tree-map list of all the important settings we need to change:
Processor Type and Features
Preemption Model (Voluntary Kernel Preemption (Desktop))
Preemptible Kernel (Low-Latency Desktop)
Enable cleancache driver to cache clean pages if tmem is present
Timer frequency (250 HZ)
1000 HZ
Bus options (PCI etc.)
PCI Stub driver
Xen PCI Frontend
Device Drivers
Block devices
Xen virtual block device support
Xen block-device backend driver
Network device support
Xen network device frontend driver
Xen backend network device
Ethernet driver support
QLogic devices
NetXen Multi port (1/10) Gigabit Ethernet NIC
Xen driver support
Xen memory balloon driver
Dynamically self-balloon kernel memory to target
Memory hotplug support for Xen balloon driver
Scrub pages before returning them to system
Xen /dev/xen/evtchn device
Backend driver support
Xen filesystem
Create compatibility mount point /proc/xen
Create xen entries under /sys/hypervisor
userspace grant access device driver
User-space grant reference allocator driver
Xen PCI-device backend driver
Firmware Drivers
EFI Variable Support via sysfs
For SSH Users
For users using ssh, you will want to use
screen
, this way if your machine goes to
sleep or your connection is lost while compiling the process will not halt in the middle.
Starting screen couldn't be easier:
screen
End For SSH Users
Now we can clean the system and then compile our kernel:
make-kpkg clean
fakeroot make-kpkg --initrd --revision=3.4.9.custom kernel_image
Note that in later versions of gcc the revision must begin with a number.
On my machine it takes roughly 20 minutes to compile a kernel, so feel free to take a
break.
Once the compilation is complete, we can back up a directory where we should now
find a portable .deb installation package. I highly recommend making a backup of this
file if you want to reinstall that kernel without all this effort in the future:
cd ..
sudo dpkg -i linux-image-3.4.9_3.4.9.custom_amd64.deb
At the end of the installation, it should automatically run
update-grub
, at which point
you can reboot your system.
Once the machine has rebooted and you are logged in, type
uname -r
and if the value
it spits out is 3.4.9 (or whatever kernel version you compiled) then you are successful!
Strap in, because the next segment will cover compiling the latest Xen from source.
References:
Amazingly Simple Kernel Compiling Guide
I have great news, the latest Xen source, RC3 currently, was the first time I have been
able to compile the source without debugging make errors since I started back in
March.
Before we proceed I should run through the fact that Xen can be affected by:
Hardware
Environment (Dom0 OS)
Software
Xen Source
Dependent Packages
Xen is a spiderweb of dependencies simply due to the number of facets it has to fill,
and we have control over barely half of these major factors.
My point is primarily that you can use the same hardware, platform, and revision I use
and still encounter brand new errors due to changes in dependent packages
downloaded during compilation. Be prepared.
The following set of packages is a bare-minimum set to compile and run Xen
successfully, and omits well over a gigabyte of the recommended dependencies:
Note: libpci-dev replaces pciutils-dev (which routes to said package), just an FYI.
sudo aptitude install bridge-utils build-essential libncurses-dev python-dev uuid uuid-dev libglib2.0-dev libyajl-dev bcc gcc-multilib iasl libpci-dev mercurial flex bison libaio-dev
These packages appear to be on the requirements list, but were already on my system
at this stage. If you have not been following from the previous segment you may want
to run this for good measure;
GUI Xen Packages
For those of us using Gnome or a GUI environment, you may want to include these
packages for vnc access and/or sdl:
sudo aptitude install libsdl-dev gvncviewer
Obviously you have the choice to use only one, you do not need both, but if you intend
to use the GUI environment in Debian I highly recommend you give SDL a spin, it's
much faster than VNC, though VNC does offer remote access.
End GUI Xen Packages
Complete The Dependencies
The list I already provided is a minimalists list, containing only exactly what was
required to compile and run Xen for my own purposes, which includes PVHVM
systems of Windows, FreeBSD, and Linux, as well as PCI Passthrough, and both
VNC and SDL. It has worked for me since I started tinkering back in March, with minor
modifications since.
For those who want the whole package and have a spare gigabyte of OS Disk space
available, I have put together this sub-section.
sudo aptitude install bcc patch libncurses5-dev python python-dev libglib2.0-dev bin86 bzip2 module-init-tools make gcc libc6-dev libcurl3 iproute xz-utils
I should also note that some of these packages will change flags marked as no from
./configure
to a yes, but that did not stop me from compiling and running the system:
zlib1g-dev
libbz2-dev
liblzo2-dev
e2fslibs-dev
ocaml
ocaml-nox
ocaml-findlib
lzma
lzma-dev
liblzma-dev
markdown
libcurl4-openssl-dev
transfig
tgif
libvncserver-dev
python-twisted
libjpeg62-dev
gawk
git-core
texinfo
texlive-latex-base
texlive-latex-recommended
texlive-fonts-extra
texlive-fonts-recommended
If anyone knows exactly what any of these dependencies do or are used for and can
shed some light on them I would greatly appreciate it.
End Complete The Dependencies
Next we can visit xenbits to get the latest revision. In our case we will use revision
25777, and use mercurial to download the source:
I actually saw revision 25784 was out, but when I tried it I got an error for a potentially
uninstantiated variable, assuming gcc compatibility bug that won't get caught for a few
more revisions.
EFI Source Modification
Given that I encountered no make errors, I was shocked to find that they still hadn't
addressed the grub efi compatibility bug where it fails to recognize available system
memory.
The fix for this is a source modification, though they have supposedly added a build
process for xen.efi, which would replace the debian.efi generated by grub. I have yet to
find adequate instructions to make that work, and even if I did may break adding
xen-
pciback.hide
to the grub configuration, which we need if we are planning to do the
easy method of PCI Passthrough.
So, let's open up
xen/arch/x86/setup.c
with our favorite text editor, and find the line
containing
( e820_raw_nr != 0 )
, and make that area look like this:
mkdir -p ~/src/xen && cd ~/src/xen
rev=25777
hg clone -r $rev http://xenbits.xen.org/hg/xen-unstable.hg/ xen-unstable.hg-rev-${rev}
cd xen-unstable.hg-rev-${rev}
#if 0
else if ( e820_raw_nr != 0 )
{
memmap_type = "Xen-e820";
}
else if ( bootsym(lowmem_kb) )
{
memmap_type = "Xen-e801";
e820_raw[0].addr = 0;
e820_raw[0].size = bootsym(lowmem_kb) << 10;
e820_raw[0].type = E820_RAM;
e820_raw[1].addr = 0x100000;
e820_raw[1].size = bootsym(highmem_kb) << 10;
e820_raw[1].type = E820_RAM;
e820_raw_nr = 2;
}
#endif
else if ( mbi->flags & MBI_MEMMAP )
Notice we are commenting out a section of code using "if 0", which always evaluates
to false, to eliminate the section of e801 mapping which breaks our ram recognition.
I have no idea what kind of adverse affects this has on other systems, but it has
worked for me since Xen 4.1.2.
End EFI Source Modification
SSH Users
Those of us using SSH to follow along should now run
screen
so we don't loose our
footing while compiling.
End SSH Users
We can now run the configuration script and compile the source:
./configure && make -j4 world && make -j4 deb
Notice the
-j4
flag which sets concurrency for us, with that setting I have compiled
Xen ins just under 7 minutes. It will consume roughly 1GB of space once fully
compiled.
Instead of
make install
I used
make deb
to create a debian package which we can
backup for future installations, and to simplify removal. This file will be placed in the
dist/
folder.
At this stage if you don't see any make errors and the debian file is in the dist folder,
you have successfully compiled Xen and are ready to move to the next phase of
installing and configuring Xen!
References:
In the previous segment we had just finished compiling Xen to the
dist/
folder, which
is where we will go now to install it!
cd dist/
sudo dpkg -i xen-upstream-4.2.0-rc4-pre.deb
Our work is not done yet, we have a lot of post-install configuration to take care of
before we can reboot and verify that Xen is working.
Let's start with a trip to
/boot
to delete extra symbolic links and related files:
cd /boot
sudo rm xen.gz xen-4.gz xen-4.2.gz xen-syms-4.2.0-rc4-pre
Next we want to have Xen boot before the Linux Dom0 kernel, Xen will load that after.
To do this we need to move the grub configuration file forward and update grub via:
sudo mv /etc/grub.d/20_linux_xen /etc/grub.d/09_linux_xen
sudo update-grub
Finally, before we reboot the system we need to make sure we initialize the Xen
toolstack and related features at boot time via:
sudo update-rc.d xencommons defaults 20 19
sudo update-rc.d xendomains defaults 21 22
sudo update-rc.d xen-watchdog defaults 23 22
At this stage we reboot our system, and if all goes well the first option in grub will be
Xen, which will bring us to our linux login.
Our next step is logging in and testing the xl toolstack with
sudo xl dmesg
, if the
output you receive is a not an error, then you have successfully installed Xen.
Next we need to setup a virtual bridge for our virtual machines going forward. Here is a
simple recommended configuration you can use to replace the contents of
/etc/network/interfaces
(See my PFSense Segment for a more complex multi-
bridge configuration):
auto lo xenbr0
iface lo inet loopback
iface eth0 inet manual
iface xenbr0 inet dhcp
bridge_ports eth0
To make this change take affect you can restart the network service, ifconfig, or take
my approach and reboot.
Making Things Easier
As many are aware, security comes at the cost of usability.
So this section is a way to make using Xen easier at the slight cost of some security.
Add these lines to the end of
/etc/sudoers
to omit entering a password when running
the xl command:
# Add XL Access
%sudo ALL=(ALL:ALL) NOPASSWD: /usr/sbin/xl
If using
vi
you may have to use
:wq!
to force write quit when saving.
Next let's add an alias so whenever we type
xl
it assumes
sudo xl
by adding an
alias to
/etc/bash.bashrc
:
# XL Alias
alias xl='sudo xl'
Now when we login next instead of using
sudo xl
and entering a password, we can
just use
xl
and no password is necessary!
End Making Things Easier
Your Xen configuration is now primed and ready to begin. You can move onto any of
these segments at your discretion:
Installing PFSense HVM
Installing Debian HVM
I recommend changing the table of contents for details on managerial activities, which
includes backing up and restoring, LVM and partitioning, and a few other helpful
related activities.
References
Outdated Document Mentions Xen Syms File
While you are welcome to follow along, I highly recommend you spend a few hours
reviewing the Xen Man Pages.
Our first step is preparing a Logical Volume partition for Xen.
With Xen, this partition will be treated as a whole hard drive, and will create a sub-
partition-table and partitions inside of it, hence you cannot simply mount the logical
volume without an offset going forward. For more information on this, check out the
managerial tasks section of my documentation.
I recommended Samba because there are drive limitations with Xen, if you intend to
share a partition between multiple machines mounting with write permissions can ruin
your data. Xen will allow multiple mounts, but neither system will inform the other of
changes, hence conflicting read and write operations.
Other disk limitations include only up to 4
hdX
drives, which are treated as Parallel
ATA, and only up to 15
sdX
drives which are treated as SCSI can be passed at any
time.
We will
You will need a Windows 7 installation DVD, I recommend a disk image copy, which
we can use to install Windows 7.
Networking is also somewhat specific, while I have not run into a problem with the mac
address myself, many others have. Xen documentation says to prefix your virtualized
mac addresses with
00:16:3e
. We will be using our virtual bridge to create a virtual
e1000 gigabit ethernet adapter for our DomU.
Here is my Windows 7 configuration file for installation:
name='windows'
builder='hvm'
vcpus=4
memory=6144
disk=[
'/dev/mapper/xen-win2,,hda,w',
'/media/samba/docs/win7.iso,,hdc,r,devtype=cdrom'
]
vif=[
'bridge=xenbr0,model=e1000,mac=00:16:3e:14:b1:1c'
]
boot='dc'
pae=1
nx=1
videoram=16
stdvga=1
sdl=1
vnc=0
usb=1
usbdevice="tablet"
localtime=1
Once Windows has been installed we can change
boot='dc'
to
boot='c'
, and
remove the iso from the drives.
Passthrough is an important and moderately complex topic. First you need to
understand what the BDF is, BDF is Bus, Device, Function, commonly in the format of
BB:DD.f
, and it is how the linux system identifies and refers to PCI devices.
To check your systems PCI device identifiers you can use
lspci
.
Once you know the BDF of the devices you wish to pass, you need to prevent Dom0
from using them via pciback. By default pciback is a module, and you can use a late-
binding method to hide a device from Dom0, but for the best experience I find a custom
kernel with pciback built-in and modified grub does the trick much better.
In my case, I have five items, the two functions of my graphics card, two USB 3.0
controllers, and a USB 2.0 controller. The identifiers are:
00:1d.0 (USB 2.0)
01:00.0 (Graphics Card Video Function)
01:00.1 (Graphics Card Audio Function)
04:00.0 (USB 3.0)
05:00.0 (USB 3.0)
If you have followed this guide, then you probably compiled a custom kernel, so our
next step is adding a line to grub!
Find this section of your grub configuration in the file
/boot/grub/grub.cfg
:
submenu "Xen 4.2.0-rc4-pre" {
menuentry 'Debian GNU/Linux, with Xen 4.2.0-rc4-pre and Linux 3.4.9' --class debian --class gnu-linux --class gnu --class os --class xen {
insmod part_gpt
The line we want to modify starts with
module
, and here is what we add:
We will have to force write-quit to save the file (
:wq!
), then when we reboot the system
our devices will be available for passthrough!
Next we add this to our HVM Configuration File:
pci = [
'00:1d.0',
'04:00.0',
'05:00.0',
'01:00.0',
'01:00.1'
]
Important Note for Graphics Passthrough
nVidia cards do not work out of the box, but ATI does.
insmod ext2
set root='(hd0,gpt2)'
search --no-floppy --fs-uuid --set=root 9b2dde6d-5888-429c-a6dd-35e7fa11f632
echo 'Loading Xen 4.2.0-rc4-pre ...'
multiboot /xen-4.2.0-rc4-pre.gz placeholder
echo 'Loading Linux 3.4.9 ...'
module /vmlinuz-3.4.9 placeholder root=/dev/mapper/xen-dom0 ro quiet
echo 'Loading initial ramdisk ...'
module /initrd.img-3.4.9
}
module /vmlinuz-3.4.9 placeholder root=/dev/mapper/xen-dom0 ro quiet xen-pciback.hide=(00:1d.0)(01:00.0)(01:00.1)(04:00.0)(05:00.0)
However, most cards do not support FLR (Function Level Reset), which is important
because it prevents the card from operating properly without the right steps.
Essentially, unlike a physical computer, when the virtual machine is shut down or
restarted, it does not change the power supplied to the graphics card. This means the
graphics card is never re-initialized, so FLR was invented. However, because most
graphics cards do not support FLR you have to mimic this effect by ejecting the card
after every reboot, otherwise you will see a severe performance degradation.
In addition, if you do not reset the card, and it is not fresh when you attempt to install or
uninstall drivers, the process may fail leaving your system crippled, plus a BSOD is
likely to appear.
So, my recommendation when dealing with GPU drivers and passthrough, always
reboot the entire machine, or take extra care to reset the card before making any
changes.
Another important fact is PV on HVM drivers, windows has GPLPV and these drivers
can dramatically improve your hard drive and network performance, among others.
After installing GPLPV drivers, you may want to add this line to your HVM
configuration:
xen_platform_pci=1
The final configuration after my devices are passed and PVHVM Drivers are installed
looks like this:
name='windows'
builder='hvm'
vcpus=4
memory=6144
disk=[
'/dev/mapper/xen-win2,,hda,w'
]
vif=[
'bridge=xenbr0,model=e1000,mac=00:16:3e:14:b1:1c'
]
pci = [
'00:1d.0',
'04:00.0',
'05:00.0',
'01:00.0',
'01:00.1'
]
boot='c'
pae=1
nx=1
videoram=16
stdvga=1
sdl=1
vnc=0
usb=1
usbdevice="tablet"
localtime=1
xen_platform_pci=1
Also, important to note that if you do not have
ntp
on Dom0, or do not set
localtime=1
in the configuration, windows will time drift at every boot, which can
cause severe problems. At one point my system would boot a month out of date, and
even with a scheduled time sync it would fail to properly reset the time. Icons would be
missing from my taskbar and SSL certificates would claim they expired. It was a scary
experience.
References
There are a few tricks I can share with regards to managing Xen virtual machines.
However, having an understanding of the fundamentals is also helpful.
Here are the key areas:
Drive Partitioning
LVM
Using mount With LVM Virtual Partitions
dd
Drive Partitioning
This is a fundamental, you are welcome to skip ahead if you want the how and not the
why.
I find a lot of emails floating around the xen-users email list due to misunderstandings
in how Xen recognizes and managed partitions.
It is important to understand how Xen Virtual Machines treat partitions.
It sees them as entire hard drives, and will create a brand new partition table and
partitions inside of it.
The result is sub-partitioning, and this causes an offset between the start of the actual
partition and the start of the sub-partition containing all of the data on the virtual
machine.
In more detail
Computers typically segment hard drives into three components.
Partition Tables
Partitions
File Systems
A partition table tells the disk where the beginning and end of a partition exists and
usually the file system.
The Partitions are simple blocks of storage.
The File System tells the operating system how to access the partition.
A normal installation of Windows for example, will look like:
Drive Block 0
Partition Table:
Starts at Block 256 File System NTFS
Drive Block 256
Partition
Data
LVM
Linux has LVM handle all the tough parts of managing the positions of data blocks,
and makes everything else easier for the user.
Hence it is called the Logical Volume Manager, because it manages logical volumes.
What this means is when we create a LVM Partition, we are actually using an LVM
"File System", which looks like this:
Drive Block 0
Partition Table:
Starts at Block 256 File System LVM
Drive Block 256
Partition
Data
We can then create partitions using LVM which automatically handles the location of
the data inside the LVM Partition.
For example, we can create two LVM Partitions for a Windows and Linux Virtual
Machine, which would look like this:
Drive Block 0
Partition Table:
Starts at Block 256 File System LVM
Drive Block 256
Partition
Data
LV Windows
LV Linux
Only an operating system that recognizes LVM can see "LV Windows" or "LV Linux"
though, so if you plug that drive into a basic Windows machine, it won't be able to do
anything with the data.
'''Finally we have Xen'''
With Xen we give one of those partitions to our virtual machine, but as stated it sees
that as a hard drive.
If we use the previous drive configuration and just add a Windows Virtual Machine to
LV Windows
our configuration looks like this:
Drive Block 0
Partition Table:
Starts at Block 256 File System LVM
Drive Block 256
Partition
Data
LV Windows
LV Block 0
Partition Table
Partition at Block 256 File System NTFS
LV Block 256
Partition
Data
LV Linux
At this point, we cannot simple access the data on our Xen partition because it has
been sub-partitioned.
If you try simply using
mount
it will fail, because it requires the offset to the start of the
NTFS sub-partition!
Using mount With LVM Virtual Partitions
To get the offset of the partition, we will use a tool like
fsdisk
or
parted
:
sudo parted /dev/mapper/xen-windows unit B print
Using
unit B
we indicate bytes which will be used by the mount command, and we
get this:
Important to note that Windows 7 creates a 100MB Boot Partition, so the first item with
the boot flag is actually the boot partition, we want the
Start
value of the second,
which is
105906176B
.
We can now mount the partition using:
Number Start End Size Type File system Flags
1 1048576B 105906175B 104857600B primary ntfs boot
2 105906176B 171796594687B 171690688512B primary ntfs
sudo mount -o offset=105906176 /dev/mapper/xen-windows /path/to/mount/dir
That is how we can access the data inside our sub-partitions.
Obviously it is ill-advised to access the data while the operating system is running,
mounted disks will NOT inform the running platform of changes, so data can get
overwritten and cause all kinds of problems.
dd
The utility
dd
is probably one of my favorites among the tools that come with most
linux/unix systems.
It can be used to create an exact copy of data, and is not subject to a number of
limitations with higher level software.
It is not the fastest utility, but supplemented with tools like gzip you can compress a
complete backup into a size similar to that you might see with professional software.
Here is a backup Example that compressed an 80GB Windows LV with 70GB of Data
into a 43GB GZipped Image File:
sudo dd if=/dev/mapper/xen-windows of=/path/to/backup/windows.img bs=1M
sudo gzip -9 windows.img
Alternative one-liner (Takes a lot longer since it has to compress "on the fly"):
sudo dd if=/dev/mapper/xen-windows bs=1M | sudo gzip -9 > /path/to/backup/windows.img.gz
I can also use this command to restore my system from that compressed image:
sudo gzip -d /path/to/backup/windows.img.gz
sudo dd if=/path/to/backup/windows.img of=/dev/mapper/xen-windows bs=1M
Once again, one-liner alternative:
There are plenty of other uses I have found for the
dd
command, but backups and
restoration make Linux amazingly easy to restore after terrible failure, and believe me
in writing this guide I encountered that plenty of times.
You've reached the end of my guide!
Thanks for taking your time to read through my documentation, if you found it at all
helpful please feel free to shoot me an email on the Xen Users email list.
Any feedback or corrections are also welcomed.
I am still working on the PFSense segment and will have that up as soon as I can.
Personal Configuration Data
This section contains documentation on my own personal configurations and is not fit
for third-party consumption.
sudo gzip -dc /path/to/backup/windows.img.gz | sudo dd of=/dev/mapper/xen-windows bs=1M
Configurations for these components will vary by user and system.
SSH Configuration
For my personal SSH configuration, I like to change the default port from 22 to
something different. This helps with security and eliminates conflicting port-forwards
for external mapping.
To do this we simply need to open up
/etc/ssh/sshd_config
and change the 22 in
port 22
to another number.
After that you need to restart
ssh
with
sudo service ssh restart
.
Samba Configuration
My Samba configuration consists of three 1TB Hard drives setup with LVM and
multiple partitions for an organized file storage system.
Samba configuration consists of:
Preparing mount points for each partitions
Setting up automatic drive mounting at boot time
Adding a samba user and changing permissions on said mount points
As of this point, Samba should already be installed, and a group called
sambashare
should exist on the system.
Our first objective will be adding our user to that group, and adding our user to the
samba users with:
sudo usermod -aG sambashare cdelorme
sudo smbpasswd -a cdelorme
When prompted for a password, this is the new password you will be able to use to
access shares over the network, and can be different than your login password.
Note: We must logout and back in before the system recognizes us as part of the
sambashare
group.
Next we need to prepare a mount point where all of our shared folders can sit, and by
default the system uses
/media
so I will create
/media/samba
:
cd /media
sudo mkdir samba
Currently I have two volume groups named
media
and
data
, and a set of volume
groups for each, my goal will be to create a set of (optionally matching) directories for
each as a mounting point:
/dev/mapper/data-backups
/dev/mapper/data-docs
/dev/mapper/data-software
/dev/mapper/media-games
/dev/mapper/media-multi
/dev/mapper/media-music
So next I run the following command to create appropriate folders:
cd /media/samba
sudo mkdir backups docs software games multi music
Now we need to think about mounting the partitions, but we need this to happen
automatically at boot time, so we turn to the
/etc/fstab
file.
This file is how we can add new partitions to mount at boot time, and we have a
decision to make.
Drives can be mounted by UUID or root path (eg.
/dev/mapper/...
). I go with UUID,
each approach has its own pros and cons.
To get the UUID of each of my partitions I use this command:
sudo blkid /dev/mapper/VolumeGroup-VolumeName
Note: No UUID will be provided if a partition has not been given a file system.
Here is what my
/etc/fstab
file contains:
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
For simplicity I reboot the system instead of running half a dozen mount commands,
which also verifies my fstab file entries are valid.
Now for one last step we just need to set permissions on our share folders and files:
sudo chown -R cdelorme:sambashare /media/samba/
Next we can finally configure Samba itself to share our folders, which is a whole
different ballgame of its own.
proc /proc proc defaults 0 0
# Originals
#/dev/mapper/xen-dom0 / ext4 defaults,noatime,discard,errors=remount-ro 0 1
#/dev/mapper/xen-home /home ext4 defaults,noatime,discard,errors=remount-ro 0 2
#/dev/mapper/xen-swap /swap swap sw 0 0
# Revised UUID Listing
UUID=da46053a-c733-4505-ab87-3f510b08e338 /boot ext4 defaults,noatime,discard,errors=remount-ro 0 2
UUID=AB5A-DB78 /boot/efi vfat utf8 0 0
UUID=dfcc1ff9-8f21-44d0-b224-447c66de7dc0 / ext4 defaults,noatime,discard,errors=remount-ro 0 1
UUID=15833a6d-488b-46ac-adfc-e1db4929d096 /home ext4 defaults,noatime,discard,errors=remount-ro 0 2
UUID=d1e67556-cf2e-4166-81b2-b9c8da10af06 none swap sw 0 0
# Samba Shared LVM Drives
UUID=0bbfb482-ca39-43c5-a1f3-39749c9dadaa /media/samba/backups ext4 defaults,errors=remount-ro 0 0
UUID=30097f44-5978-4fc4-b5cf-663dca259363 /media/samba/docs ext4 defaults,errors=remount-ro 0 0
UUID=4fe0435a-7041-402e-abb3-708e258d39cd /media/samba/software ext4 defaults,errors=remount-ro 0 0
UUID=6c612d46-f0b9-405f-8a62-f77bdc266237 /media/samba/music ext4 defaults,errors=remount-ro 0 0
UUID=3f9f7128-d004-4b1a-81d7-e752631127a1 /media/samba/multi ext4 defaults,errors=remount-ro 0 0
UUID=b56f7d04-6037-4bfe-854e-5f7ee9a7da4a /media/samba/games ext4 defaults,errors=remount-ro 0 0
# DVDRom
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0
My documentation is scarce, because the default samba configuration file (located at
/etc/samba/smb.conf
) had terrible default settings that shared my entire home folder,
so I had to gut it, and after much online reading and a bit of testing I ended up with this
simplified version:
#======================= Global Settings =======================
[global]
workgroup = WORKGROUP
server string = %h server
dns proxy = no
log file = /var/log/samba/log.%m
max log size = 1000
syslog = 0
panic action = /usr/share/samba/panic-action %d
encrypt passwords = true
passdb backend = tdbsam
obey pam restrictions = yes
unix password sync = yes
passwd program = /usr/bin/passwd %u
passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
pam password change = yes
map to guest = bad user
usershare allow guests = yes
#======================= Share Definitions =======================
[Backups]
create mask = 660
directory mask = 770
path = /media/samba/backups
write list = cdelorme
[Docs]
create mask = 660
directory mask = 770
Note: For write and read lists, to specify a group, prefix with
@
, otherwise use a
username. The create and directory masks are just defaults I found in many examples.
After that file is saved I simply restart samba with
sudo service samba restart
and
voila everytime the system boots it will automatically share my folders, which can now
be accessed over the network by any operating system and with excellent speeds.
I have measured speeds in excess of 100MB over the network from Xen virtual
path = /media/samba/docs
write list = cdelorme
[Software]
create mask = 660
directory mask = 770
path = /media/samba/software
read list = @sambashare
write list = cdelorme
[Music]
create mask = 660
directory mask = 770
path = /media/samba/music
write list = @sambashare
[Games]
create mask = 660
directory mask = 770
path = /media/samba/games
read list = @sambashare
write list = cdelorme
[Multi]
create mask = 660
directory mask = 770
path = /media/samba/multi
read list = @sambashare
write list = cdelorme
machines, without interruptions to other running network services.
References
Input Reference
Keyboard Reference
Mount SFTP in File Manager
Get Aptitude Package History
End
Wyszukiwarka
Podobne podstrony:
Comprehensive Debian (pion)
instalacja debiana
ANALIZA PRZYCZYN WYBUCHU WYBRANEJ WOJNY NA 3 POZIOMACH
Cw 4 poziomy komunikacji
METODY BADANIA PRACY poziome 2
1 Profilaktyka spoleczna pojecie, poziomy, kategorie, strategieid 9645 ppt
arkusz fizyka poziom s rok 2001 535
Odpowiedzi do testu 1 poziom podstawowy id 332449
Lubelska Próba Przed Maturą Marzec 2015 GR B Poziom Rozszerzony
KLASA 1 POZIOM ROZSZERZONY doc Nieznany
arkusz Matematyka poziom r rok 2010 4393 MODEL
PJM Poziom A2 Strona 90
więcej podobnych podstron