Comprehensive Debian (pion)

background image

Comprehensive Debian
Wheezy Xen Tutorial

Table of Contents

Introduction

Objectives Overview

UEFI Configuration

Wheezy Installation
Wheezy Configuration

Compiling a Custom Linux Kernel
Compiling Xen

Xen Configuration

Installing Windows HVM

Installing PFSense HVM

Installing Debian HVM

Managerial Tasks

Introduction

The following guide is comprised of my personal

documentation and excessive filtering for human

consumption. It details the series of steps I took to

successfully install and compile a custom Linux Kernel,

Xen 4.2 unstable, and run three virtual machines to

perform unique tasks all on a single physical computer.

To achieve the desired functionality, I used IOMMU for
PCI Passthrough with a multimedia operating system,

granting complete access to a graphics card for GPU
Acceleration.

background image

I am a college student, not a trained professional, and I
am sharing this documentation for educational purposes.
Blind use of this document for a production environment
in a business setting would be ill-advised.

To quickly summarize my experiences, I decided to try
Xen and began researching in early January 2012.

I purchased equipment in March 2012, and began what I
thought would take "at most two weeks".

By late April I finally had a (mostly) functional system.

I have been fine tuning the system and my process since
then to produce this guide for others.

The purpose of this guide is to turn a 6 month project into
a series of steps that can be reproduced inside the time

frame of 1-2 days.

Objectives Overview

I wanted a single physical computer, that could handle

three computers worth of separated activities at all times,

including these four specifics:

Router/Firewall (PFSense)

DNS & Web Development (Debian Squeeze)

Application Development (Debian Wheezy)

Multimedia & Gaming (Windows 7)

I investigated alternative software, including VMWare's

ESXi and Citrix's XenServer.

I had come from a VMWare platform having used both

VMware Server 2 and VMWare Workstation 8 for the
same key objectives previously but with a Windows

background image

Host, subject to Windows Updates which let to my

investigation.

ESXi was easy to install, but missing numerous drivers

for hardware components. I quickly ruled it out as I

wanted a home-use system, and there was no
convenient or well-documented method to installing or

even launching a virtual machine from their on-server
interface.

Citrix's XenServer was easily my favorite of the options,

it's design made for a great user experience. Everything
was easy to find and do. However it came with a

minimum price tag of $1000, and if I wanted PCI
passthrough for graphics cards $2500. This was beyond

my reach as a college student.

I chose Xen since it had support for my hardware, was a

free open-source project, and had a great community
with lots of activity.

Things you Will Need:

Compatible Hardware

ATI Graphics Card
Motherboard with UEFI & VT-d & Onboard

Graphics
Latest Ubuntu Live DVD

Debian Wheezy Beta1 (Or Newer) Installer

nVidia Cards can be made to work, with extensive
patching, in Windows XP and supposedly Windows 8

Preview, for more information visit

David Techer's blog

.

If you plan to pass your graphics card to a virtual
machine, you will need either a second graphics card,

onboard graphics, or a second computer to manage your

Dom0 system and install virtual machines over VNC.

background image

UEFI compatible boot DVD will save you an
undocumented step for setting up a UEFI boot loader.

Xen is picky, different hardware may yield different

results, both at compile time and runtime. If you want to
save yourself some hassle, here is my hardware list and

some suggestions to avoid:

Motherboard:

ASRock Z68 Extreme4 Gen3

CPU:

Intel Core i2600

RAM:

12GB 1333Mhz Corsair XMS (2x2G + 2x4G)

Boot Disk:

240GB OCZ Vertex 3

GPU:

ATI Radeon HD 6870

LAN:

Onboard Broadcom BCM57781

PCIe EXPI9301CTBLK

Hardware and Configurations to Avoid:

NF200 Chipsets are not IOMMU compatible

nVidia Graphics Cards

RAID5 yields horrible performance

NF200 is a PCI Switch for motherboards sporting SLI
and CrossFire, avoid it if you want be able to use those
PCI slots for passthrough.

UEFI Configuration

These instructions are for an Intel CPU and ASRock

background image

UEFI Motherboard, and may vary depending on your
manufacturer as well as your choice of CPU. If you are
unfamiliar with motherboard configuration, you may want
to watch the video for a visual walk through.

Before starting reset your CMOS so you have a clean

slate to work from.

List of key settings:

Turn Legacy USB 3.0 off

Turn on VT-x and VT-d

Set drives to ACHI Mode
Change default video to Onboard

Justification:

Consider yourself warned, if you leave Legacy USB on

and leave a backup USB drive connected, your system

will fail to boot. To my surprise that is because it is trying

to boot from that USB drive (even if that drive is USB
2.0
), which threw me off so I recommend disabling that

feature.

Most motherboards will have Intel Virtualization enabled

by default (VT-x), but that is not the same as IOMMU, be

sure to look for VT-d or check the manual for details. For

my system VT-x was in the CPU configuration area, and
VT-d was in the Northbridge configuration area.

ACHI is a superior choice for performance with modern

hard drives, and while RAID is an alternative that uses

ACHI as its underlying type, it increases boot time by

checking RAID configurations, and most onboard RAID
is software RAID that is built for Windows and rarely

helpful for Linux.

ASRock boards will use a PCI GPU by default if it is

plugged into a PCI slot, regardless of if any video cables

are connected, and if you plan to pass that device to a

background image

virtual machine, you do not want to use it for video.

Wheezy Installation

Select Advanced Options from the menu, and then

Advanced Install.

To expedite this process we can select defaults for most

options (such as for networking).

Once we reach Hard Drive Partitioning options, select

Manual at the bottom.

Select the disk(s) we will be using by title not partition
space and hit enter, we can now select gpt from the

partition tables instead of msdos.

We will then create three partitions:

256MB FAT32 Partition mounted to /boot/efi

256MB Ext4 Partition mounted to /boot

Remainder LVM Partition

Select Configure the Logical Volume Manager and
create a volume group.

Create three Logical Volumes:

8GB or More for Linux

20B or More for Home

2GB or More for Swap

Complete LVM Configuration, and then back to our

partition settings we can now select formats and mount

points:

8GB LV Ext4 for root (/)

20GB LV Ext4 for /home

background image

2GB LV swap

Optionally add noatime flags to all the Ext4 partitioned
disks.

Complete the Partition steps by selecting done and

create the partitions.

Continue through the options with defaults, until we get

to the main Packages.

De-select Debian Desktop Environment and

continue

Do not install a boot loader, select the Continue
without a bootloader
option and be sure to jott down

the information it provides us for later.

Complete the installation and restart our computer.

Insert the Ubuntu Live DVD and press whatever key

opens the boot options menu at startup.

Select the UEFI boot option from that menu.

When presented with an options screen, press c on the

keyboard to access the Emergency Grub Menu.

Emergency Grub allows us to manually boot our Linux,
since we did not install a boot loader.

Type ls to list the disks, we should have an (hd0) and

three gpt partitions.

Assuming the same partitioning as mine, in the below

example tab indicates hit the tab key to auto-complete,

and we can either use the data provided when we
selected Continue without a bootloader or just

substitute xen for the name we gave our Volume Group

name, and linux for our root (/)Logical Volume:

background image

Gibberish may be displayed when tabbing to auto-

complete, this is normal and can be ignored. After
entering boot it will begin starting Linux.

If done successfully a terminal login will be presented,

which concludes this segment of the guide.

In the next segment we will start by enter our credentials

and we can start installing some key packages to get

UEFI working.

References

EFI Grub on Wheezy

Wheezy Configuration

Note:

I have done my best to separate my crazier desires from
the average persons needs.

To do this I created bold headers with line breaks for

special sub-sections, whose subsequent contents you
can skip if said bold text does not apply to your situation.

Assuming you have just installed Debian Wheezy and

are at the login prompt, you want to login as root so you
can install packages (alternatively you can use

su

to

grub> set root=(hd0,gpt2)
grub> linux /vmlinu_tab_ ro root=/dev/mapper/xen-linux
grub> initrd /initrd_tab_
grub> boot

background image

change users if you prefer).

UEFI STEPS

Assuming you came from my previous guide and have
just manually booted, your first objective should be

getting an efi bootloader installed:

aptitude install grub-efi-amd64

Once installed you will want to run the install and update

scripts:

grub-install
update-grub

END UEFI STEPS

Now let's install a couple more basic packages:

aptitude install ssh sudo

With ssh installed we can, optionlly, access remotely for
the remainder of the guide. With sudo we can add our
user to the sudoers group like so (cdelorme is my

username):

usermod -aG sudo cdelorme

If you omit the

a

you will replace all existing groups your

user belongs to, so be careful.

I recommend rebooting now to get ssh working, and also
so you can login as your user to use the sudo command

(if you used

su

you may need to logout and log back in

for

sudo

to work).

Next we want to prime our server with some general

background image

utility packages:

parted

samba
ntp

screen
p7zip-full

Delicious Copy Pasta:

Note: I used to install

ntfsprogs

but the package is

currently bugged and eats CPU cycles, so I would

avoid it if possible.

Configuration of samba and ntp will vary by system, I
recommend using other guides online for this since as
long as they work they will be fine when we start using

Xen. If you want my configurations specifically, ask and I
may consider adding them to a future edition of this
guide.

Development Environment Steps

Since I intend to use Debian Wheezy as a development
environment, I also installed these packages:

openjdk-7-jdk

ant
g++
libboost-dev

gnustep
python
python-dev

Delicious Copy Pasta:

sudo aptitude install parted samba ntp screen p7zip-full

sudo aptitude install openjdk-7-jdk ant g++ libboost-dev gnustep python python-dev

background image

End Development Environment Steps

GUI Environment Steps

To be honest, I don't care for the CD Installation of
Gnome3, it's less work but the cost is 30 minutes of
installation time, and a whole slew of packages I never

use, so I went minimalist and found exactly what I
needed:

gnome-session

gnome-terminal
gnome-screensaver
xserver-xorg-core

gdm3
gksu

ia32-libs
ia32-libs-gtk
libc6-dev-i386

binfmt-support

Delicious Copy Pasta:

If you intend to use Xen with SDL later on, I recommend

replacing

xserver-xorg-core

with

xorg

and

xorg-dev

packages, as these are what we will install later, and

they contain the

xserver-xorg-core

.

As a bare-bones install of Gnome3, it should take
roughly 5 minutes as opposed to 30.

At this stage you can run

startx

to launch the GUI

environment, but I generally reboot to test the login

screen.

End GUI Environment Steps

sudo aptitude install gnome-session gnome-terminal gnome-screensaver xserver-xorg-core gdm3 gksu ia32-libs ia32-libs-gtk libc6-dev-i386 binfmt-support

background image

Optional GUI Additions

Optional additions include enabling audio, installing

some basic multimedia, utilities, and important software
like Guake terminal, Google Chrome, and Sublime Text.

Starting with Audio:

sudo aptitude install alsa-base
sudo alsactl init

Now with GUI utilities:

python-xdg
font-manager

gnome-disk-utility
gnome-screenshot
guake

Delicious Copy Pasta:

Note: Guake has a bug with Gnome3 that prevents it

from starting at boot, here is how you can fix it:

This is actually a very simple edit, in this specific version
of guake (located at

/usr/bin/guake

) line 653 contains

a

notification.show()

, we just have to wrap it in a

simple try-catch so it doesn't crash when it is run before
the GUI is ready to accept a pynotify event:

try:
notification.show()
except Exception:
pass

Next we want to set Guake to start at login. You can

make an auto-start file via:

sudo aptitude install python-xdg font-manager gnome-disk-utility gnome-screenshot guake

background image

For google chrome we need to add a new debian source

to

/etc/apt/sources.list

, and download their package

key:

Next we can install these packages:

google-chrome-unstable
eog
gnash

vlc
gtk-recordmydesktop

Delicious Copy Pasta:

Note: I am aware that google source is for Squeeze, I
have had no problems running it in Wheezy.

After google-chrome has been installed, it will create its

own sources.list file entry, so we want to go back in and
remove the line we added from

/etc/apt/sources.list

,

mkdir ~/.config/autostart
cd ~/.config/autostart
echo "[Desktop Entry]" >> guake.desktop
echo "Name=Guake Terminal" >> guake.desktop
echo "Name[pt]=Guake Terminal" >> guake.desktop
echo "Name[pt_BR]=Guake Terminal" >> guake.desktop
echo "Comment=Use the command line in a Quake-like terminal" >> guake.desktop
echo "TryExec=guake" >> guake.desktop
echo "Exec=guake" >> guake.desktop
echo "Icon=/usr/share/pixmaps/guake/guake.png" >> guake.desktop
echo "Type=Application" >> guake.desktop
echo "Categories=GNOME;GTK;Utility;TerminalEmulator;" >> guake.desktop

su root
echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
aptitude update
exit

sudo aptitude install google-chrome-unstable eog gnash vlc gtk-recordmydesktop

background image

and re-run

sudo aptitude update

to make sure it works.

Finally, I am in love with Sublime Text 2, so I went to
their website for the download, and went through these

steps for a complete installation.

Once downloaded, let's extract it and move it to

/usr/local

, and create a new command to run it from

terminal:

Next let's create a desktop file so we can run it as an
application and set default mimetypes from the GUI.

Let's navigate to

/usr/share/applications/

and create

a new file

subl.desktop

with these contents:

Next to set the mimetypes we add these lines to

/usr/share/applications/defaults.list

:

text/plain=subl.desktop
text/css=subl.desktop
text/htm=subl.desktop
text/javascript=subl.desktop
text/x-c=subl.desktop
text/csv=subl.desktop
text/x-java-source=subl.desktop
text/java=subl.desktop

There are user-account alternatives that may seem more

tar xf sublime-text.tar.bz2
sudo mv sublime-text /usr/local/
cd /usr/local/sublime-text
sudo ln -s /usr/local/sublime-text/sublime_text /usr/bin/subl

[Desktop Entry]
Name=Sublime Text 2
Comment=The Best Text Editor in the World!
TryExec=subl
Exec=subl
Icon=/usr/local/sublime-text/Icon/256x256/sublime_text.png
Type=Application

background image

appropriate, but I wanted these to apply system-wide.

End Optional GUI Additions

Thus concludes our Debian Wheezy configuration, and

we are now ready to move onto compiling a Custom
Linux Kernel.

References

Missing ia32-libs-gtk for Gnome3

Samba Config

Compiling a Custom Linux
Kernel

Compiling a custom Linux Kernel will make PCI

passthrough much easier, but it also allows us to
improve performance in various areas for Dom0.

For starters let's create a folder for compiling our

packages in our home directory:

mkdir -p ~/src/linux
cd ~/src/linux

Next we will install the required packages:

wget

tar
bzip2

build-essential

background image

libncurses-dev
kernel-package

fakeroot

Delicious copy pasta:

To improve compiling speed we can set the concurrency
to double the number of physical CPU cores by running

these commands:

IMPORTANT NOTE: Kernel 3.5 and 3.5.2 are bugged
and passthrough with USB Controllers fails (though
VGA Passthrough works for some odd reason). That is
why this guide is using 3.4.9, however 3.3.8 has also
been tested and works.

Our next step is downloading, extracting, and inserting

the current kernel configuration before adjusting the
kernel we are about to compile:

Configuring a kernel is not as complex as it seems, the
menu is just a hierarchy of options, and thankfully you
can search for settings with

/

and get details on

individual settings with

h

.

Here is a tree-map list of all the important settings we

need to change:

sudo aptitude install wget tar bzip2 build-essential libncurses-dev kernel-package fakeroot

su root
echo "CONCURRENCY_LEVEL=8" >> /etc/kernel-pkg.conf
exit

wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.4.9.tar.bz2
tar jxf linux-3.4.9.tar.bz2 && rm linux-3.4.9.tar.bz2
cd linux-3.4.9/
cp /boot/config-3.2.0-3-amd64 .config
make menuconfig

background image

Processor Type and Features

Preemption Model (Voluntary Kernel
Preemption (Desktop))

Preemptible Kernel (Low-Latency

Desktop)

Enable cleancache driver to cache clean
pages if tmem is present
Timer frequency (250 HZ)

1000 HZ

Bus options (PCI etc.)

PCI Stub driver
Xen PCI Frontend

Device Drivers

Block devices

Xen virtual block device support
Xen block-device backend driver

Network device support

Xen network device frontend driver
Xen backend network device

Ethernet driver support

QLogic devices

NetXen Multi port (1/10)
Gigabit Ethernet NIC

Xen driver support

Xen memory balloon driver
Dynamically self-balloon kernel
memory to target
Memory hotplug support for Xen
balloon driver

Scrub pages before returning them to
system
Xen /dev/xen/evtchn device
Backend driver support
Xen filesystem

Create compatibility mount point
/proc/xen
Create xen entries under

background image

/sys/hypervisor
userspace grant access device driver
User-space grant reference allocator
driver

Xen PCI-device backend driver

Firmware Drivers

EFI Variable Support via sysfs

For SSH Users

For users using ssh, you will want to use

screen

, this

way if your machine goes to sleep or your connection is
lost while compiling the process will not halt in the
middle.

Starting screen couldn't be easier:

screen

End For SSH Users

Now we can clean the system and then compile our
kernel:

Note that in later versions of gcc the revision must begin

with a number.

On my machine it takes roughly 20 minutes to compile a
kernel, so feel free to take a break.

Once the compilation is complete, we can back up a
directory where we should now find a portable .deb
installation package. I highly recommend making a
backup of this file if you want to reinstall that kernel
without all this effort in the future:

make-kpkg clean
fakeroot make-kpkg --initrd --revision=3.4.9.custom kernel_image

background image

At the end of the installation, it should automatically run

update-grub

, at which point you can reboot your

system.

Once the machine has rebooted and you are logged in,
type

uname -r

and if the value it spits out is 3.4.9 (or

whatever kernel version you compiled) then you are
successful!

Strap in, because the next segment will cover compiling
the latest Xen from source.

References:

Kernel Source

Amazingly Simple Kernel Compiling Guide

Kernel Flag Database for Lookup

Performance Regarding Paravirt Spinlocks

Compiling Xen

I have great news, the latest Xen source, RC3 currently,

was the first time I have been able to compile the source
without debugging make errors since I started back in
March.

Before we proceed I should run through the fact that Xen
can be affected by:

cd ..
sudo dpkg -i linux-image-3.4.9_3.4.9.custom_amd64.deb

background image

Hardware
Environment (Dom0 OS)
Software
Xen Source

Dependent Packages

Xen is a spiderweb of dependencies simply due to the
number of facets it has to fill, and we have control over
barely half of these major factors.

My point is primarily that you can use the same
hardware, platform, and revision I use and still encounter
brand new errors due to changes in dependent
packages downloaded during compilation. Be prepared.

The following set of packages is a bare-minimum set to
compile and run Xen successfully, and omits well over a

gigabyte of the recommended dependencies:

Note: libpci-dev replaces pciutils-dev (which routes to

said package), just an FYI.

These packages appear to be on the requirements list,
but were already on my system at this stage. If you have
not been following from the previous segment you may
want to run this for good measure;

GUI Xen Packages

For those of us using Gnome or a GUI environment, you
may want to include these packages for vnc access
and/or sdl:

sudo aptitude install libsdl-dev gvncviewer

sudo aptitude install bridge-utils build-essential libncurses-dev python-dev uuid uuid-dev libglib2.0-dev libyajl-dev bcc gcc-multilib iasl libpci-dev mercurial flex bison libaio-dev

sudo aptitude install bcc patch libncurses5-dev python python-dev libglib2.0-dev bin86 bzip2 module-init-tools make gcc libc6-dev libcurl3 iproute xz-utils

background image

Obviously you have the choice to use only one, you do
not need both, but if you intend to use the GUI
environment in Debian I highly recommend you give

SDL a spin, it's much faster than VNC, though VNC
does offer remote access.

End GUI Xen Packages

Complete The Dependencies

The list I already provided is a minimalists list,
containing only exactly what was required to compile
and run Xen for my own purposes, which includes
PVHVM systems of Windows, FreeBSD, and Linux, as
well as PCI Passthrough, and both VNC and SDL. It has
worked for me since I started tinkering back in March,

with minor modifications since.

For those who want the whole package and have a
spare gigabyte of OS Disk space available, I have put
together this sub-section.

I should also note that some of these packages will
change flags marked as no from

./configure

to a yes,

but that did not stop me from compiling and running the

system:

zlib1g-dev
libbz2-dev
liblzo2-dev
e2fslibs-dev
ocaml

ocaml-nox
ocaml-findlib
lzma
lzma-dev
liblzma-dev

markdown

background image

libcurl4-openssl-dev
transfig
tgif
libvncserver-dev
python-twisted

libjpeg62-dev
gawk
git-core
texinfo
texlive-latex-base

texlive-latex-recommended
texlive-fonts-extra
texlive-fonts-recommended

If anyone knows exactly what any of these
dependencies do or are used for and can shed some
light on them I would greatly appreciate it.

End Complete The Dependencies

Next we can visit xenbits to get the latest revision. In our
case we will use revision 25777, and use mercurial to
download the source:

I actually saw revision 25784 was out, but when I tried it I
got an error for a potentially uninstantiated variable,
assuming gcc compatibility bug that won't get caught for

a few more revisions.

EFI Source Modification

Given that I encountered no make errors, I was shocked
to find that they still hadn't addressed the grub efi

mkdir -p ~/src/xen && cd ~/src/xen
rev=25777
hg clone -r $rev http://xenbits.xen.org/hg/xen-unstable.hg/ xen-unstable.hg-rev-${rev}
cd xen-unstable.hg-rev-${rev}

background image

compatibility bug where it fails to recognize available
system memory.

The fix for this is a source modification, though they have
supposedly added a build process for xen.efi, which
would replace the debian.efi generated by grub. I have
yet to find adequate instructions to make that work, and

even if I did may break adding

xen-pciback.hide

to the

grub configuration, which we need if we are planning to
do the easy method of PCI Passthrough.

So, let's open up

xen/arch/x86/setup.c

with our

favorite text editor, and find the line containing

(

e820_raw_nr != 0 )

, and make that area look like this:

Notice we are commenting out a section of code using "if
0", which always evaluates to false, to eliminate the
section of e801 mapping which breaks our ram
recognition.

I have no idea what kind of adverse affects this has on

other systems, but it has worked for me since Xen 4.1.2.

#if 0
else if ( e820_raw_nr != 0 )
{
memmap_type = "Xen-e820";
}
else if ( bootsym(lowmem_kb) )
{
memmap_type = "Xen-e801";
e820_raw[0].addr = 0;
e820_raw[0].size = bootsym(lowmem_kb) << 10;
e820_raw[0].type = E820_RAM;
e820_raw[1].addr = 0x100000;
e820_raw[1].size = bootsym(highmem_kb) << 10;
e820_raw[1].type = E820_RAM;
e820_raw_nr = 2;
}
#endif
else if ( mbi->flags & MBI_MEMMAP )

background image

End EFI Source Modification

SSH Users

Those of us using SSH to follow along should now run

screen

so we don't loose our footing while compiling.

End SSH Users

We can now run the configuration script and compile the
source:

./configure && make -j4 world && make -j4 deb

Notice the

-j4

flag which sets concurrency for us, with

that setting I have compiled Xen ins just under 7
minutes. It will consume roughly 1GB of space once fully

compiled.

Instead of

make install

I used

make deb

to create a

debian package which we can backup for future
installations, and to simplify removal. This file will be
placed in the

dist/

folder.

At this stage if you don't see any make errors and the
debian file is in the dist folder, you have successfully

compiled Xen and are ready to move to the next phase
of installing and configuring Xen!

References:

Xenbits Repositories

Latest Instructions

4.1 Compilation Instructions

background image

4.0 Compilation Instructions

David Techer's Blog

EFI Memory Error

Xen Configuration

In the previous segment we had just finished compiling

Xen to the

dist/

folder, which is where we will go now

to install it!

cd dist/
sudo dpkg -i xen-upstream-4.2.0-rc4-pre.deb

Our work is not done yet, we have a lot of post-install
configuration to take care of before we can reboot and

verify that Xen is working.

Let's start with a trip to

/boot

to delete extra symbolic

links and related files:

Next we want to have Xen boot before the Linux Dom0
kernel, Xen will load that after. To do this we need to

move the grub configuration file forward and update grub
via:

Finally, before we reboot the system we need to make
sure we initialize the Xen toolstack and related features

at boot time via:

cd /boot
sudo rm xen.gz xen-4.gz xen-4.2.gz xen-syms-4.2.0-rc4-pre

sudo mv /etc/grub.d/20_linux_xen /etc/grub.d/09_linux_xen
sudo update-grub

background image

sudo update-rc.d xencommons defaults 20 19
sudo update-rc.d xendomains defaults 21 22
sudo update-rc.d xen-watchdog defaults 23 22

At this stage we reboot our system, and if all goes well
the first option in grub will be Xen, which will bring us to
our linux login.

Our next step is logging in and testing the xl toolstack
with

sudo xl dmesg

, if the output you receive is a not an

error, then you have successfully installed Xen.

Next we need to setup a virtual bridge for our virtual

machines going forward. Here is a simple recommended
configuration you can use to replace the contents of

/etc/network/interfaces

(See my PFSense Segment

for a more complex multi-bridge configuration):

auto lo xenbr0
iface lo inet loopback

iface eth0 inet manual
iface xenbr0 inet dhcp
bridge_ports eth0

To make this change take affect you can restart the
network service, ifconfig, or take my approach and
reboot.

Making Things Easier

As many are aware, security comes at the cost of
usability.

So this section is a way to make using Xen easier at the
slight cost of some security.

Add these lines to the end of

/etc/sudoers

to omit

entering a password when running the xl command:

background image

# Add XL Access
%sudo ALL=(ALL:ALL) NOPASSWD: /usr/sbin/xl

If using

vi

you may have to use

:wq!

to force write quit

when saving.

Next let's add an alias so whenever we type

xl

it

assumes

sudo xl

by adding an alias to

/etc/bash.bashrc

:

# XL Alias
alias xl='sudo xl'

Now when we login next instead of using

sudo xl

and

entering a password, we can just use

xl

and no

password is necessary!

End Making Things Easier

Your Xen configuration is now primed and ready to
begin. You can move onto any of these segments at your
discretion:

Installing Windows HVM
Installing PFSense HVM

Installing Debian HVM

I recommend changing the table of contents for details
on managerial activities, which includes backing up and
restoring, LVM and partitioning, and a few other helpful
related activities.

References

Xen Man Pages

Outdated Document Mentions Xen Syms File

background image

Installing Windows HVM

While you are welcome to follow along, I highly
recommend you spend a few hours reviewing the Xen

Man Pages.

Our first step is preparing a Logical Volume partition for
Xen.

With Xen, this partition will be treated as a whole hard
drive, and will create a sub-partition-table and partitions
inside of it, hence you cannot simply mount the logical
volume without an offset going forward. For more

information on this, check out the managerial tasks
section of my documentation.

I recommended Samba because there are drive
limitations with Xen, if you intend to share a partition
between multiple machines mounting with write
permissions can ruin your data. Xen will allow multiple

mounts, but neither system will inform the other of
changes, hence conflicting read and write operations.

Other disk limitations include only up to 4

hdX

drives,

which are treated as Parallel ATA, and only up to 15

sdX

drives which are treated as SCSI can be passed at

any time.

We will

You will need a Windows 7 installation DVD, I
recommend a disk image copy, which we can use to

install Windows 7.

Networking is also somewhat specific, while I have not
run into a problem with the mac address myself, many
others have. Xen documentation says to prefix your

background image

virtualized mac addresses with

00:16:3e

. We will be

using our virtual bridge to create a virtual e1000 gigabit

ethernet adapter for our DomU.

Here is my Windows 7 configuration file for installation:

Once Windows has been installed we can change

boot='dc'

to

boot='c'

, and remove the iso from the

drives.

Passthrough is an important and moderately complex
topic. First you need to understand what the BDF is,
BDF is Bus, Device, Function, commonly in the format of

BB:DD.f

, and it is how the linux system identifies and

refers to PCI devices.

To check your systems PCI device identifiers you can
use

lspci

.

Once you know the BDF of the devices you wish to

name='windows'
builder='hvm'
vcpus=4
memory=6144
disk=[
'/dev/mapper/xen-win2,,hda,w',
'/media/samba/docs/win7.iso,,hdc,r,devtype=cdrom'
]
vif=[
'bridge=xenbr0,model=e1000,mac=00:16:3e:14:b1:1c'
]
boot='dc'
pae=1
nx=1
videoram=16
stdvga=1
sdl=1
vnc=0
usb=1
usbdevice="tablet"
localtime=1

background image

pass, you need to prevent Dom0 from using them via
pciback. By default pciback is a module, and you can

use a late-binding method to hide a device from Dom0,
but for the best experience I find a custom kernel with
pciback built-in and modified grub does the trick much
better.

In my case, I have five items, the two functions of my
graphics card, two USB 3.0 controllers, and a USB 2.0

controller. The identifiers are:

00:1d.0 (USB 2.0)
01:00.0 (Graphics Card Video Function)
01:00.1 (Graphics Card Audio Function)
04:00.0 (USB 3.0)
05:00.0 (USB 3.0)

If you have followed this guide, then you probably

compiled a custom kernel, so our next step is adding a
line to grub!

Find this section of your grub configuration in the file

/boot/grub/grub.cfg

:

The line we want to modify starts with

module

, and here

is what we add:

submenu "Xen 4.2.0-rc4-pre" {
menuentry 'Debian GNU/Linux, with Xen 4.2.0-rc4-pre and Linux 3.4.9' --class debian --class gnu-linux --class gnu --class os --class xen {
insmod part_gpt
insmod ext2
set root='(hd0,gpt2)'
search --no-floppy --fs-uuid --set=root 9b2dde6d-5888-429c-a6dd-35e7fa11f632
echo 'Loading Xen 4.2.0-rc4-pre ...'
multiboot /xen-4.2.0-rc4-pre.gz placeholder
echo 'Loading Linux 3.4.9 ...'
module /vmlinuz-3.4.9 placeholder root=/dev/mapper/xen-dom0 ro quiet
echo 'Loading initial ramdisk ...'
module /initrd.img-3.4.9
}

background image

We will have to force write-quit to save the file (

:wq!

),

then when we reboot the system our devices will be
available for passthrough!

Next we add this to our HVM Configuration File:

pci = [
'00:1d.0',
'04:00.0',
'05:00.0',
'01:00.0',
'01:00.1'
]

Important Note for Graphics Passthrough

nVidia cards do not work out of the box, but ATI does.

However, most cards do not support FLR (Function
Level Reset), which is important because it prevents the
card from operating properly without the right steps.

Essentially, unlike a physical computer, when the virtual

machine is shut down or restarted, it does not change
the power supplied to the graphics card. This means the
graphics card is never re-initialized, so FLR was
invented. However, because most graphics cards do not
support FLR you have to mimic this effect by ejecting the
card after every reboot, otherwise you will see a severe
performance degradation.

In addition, if you do not reset the card, and it is not fresh
when you attempt to install or uninstall drivers, the
process may fail leaving your system crippled, plus a
BSOD is likely to appear.

So, my recommendation when dealing with GPU drivers

and passthrough, always reboot the entire machine, or

module /vmlinuz-3.4.9 placeholder root=/dev/mapper/xen-dom0 ro quiet xen-pciback.hide=(00:1d.0)(01:00.0)(01:00.1)(04:00.0)(05:00.0)

background image

take extra care to reset the card before making any
changes.

Another important fact is PV on HVM drivers, windows

has GPLPV and these drivers can dramatically improve
your hard drive and network performance, among others.

After installing GPLPV drivers, you may want to add this
line to your HVM configuration:

xen_platform_pci=1

The final configuration after my devices are passed and
PVHVM Drivers are installed looks like this:

name='windows'
builder='hvm'
vcpus=4
memory=6144
disk=[
'/dev/mapper/xen-win2,,hda,w'
]
vif=[
'bridge=xenbr0,model=e1000,mac=00:16:3e:14:b1:1c'
]
pci = [
'00:1d.0',
'04:00.0',
'05:00.0',
'01:00.0',
'01:00.1'
]
boot='c'
pae=1
nx=1
videoram=16
stdvga=1
sdl=1
vnc=0
usb=1
usbdevice="tablet"
localtime=1
xen_platform_pci=1

background image

Also, important to note that if you do not have

ntp

on

Dom0, or do not set

localtime=1

in the configuration,

windows will time drift at every boot, which can cause
severe problems. At one point my system would boot a
month out of date, and even with a scheduled time sync
it would fail to properly reset the time. Icons would be
missing from my taskbar and SSL certificates would
claim they expired. It was a scary experience.

References

Xen Man Pages

Managerial Activities

There are a few tricks I can share with regards to
managing Xen virtual machines.

However, having an understanding of the fundamentals

is also helpful.

Here are the key areas:

Drive Partitioning
LVM
Using mount With LVM Virtual Partitions
dd

Drive Partitioning

This is a fundamental, you are welcome to skip ahead if
you want the how and not the why.

background image

I find a lot of emails floating around the xen-users email
list due to misunderstandings in how Xen recognizes
and managed partitions.

It is important to understand how Xen Virtual Machines
treat partitions.

It sees them as entire hard drives, and will create a
brand new partition table and partitions inside of it.

The result is sub-partitioning, and this causes an offset
between the start of the actual partition and the start of

the sub-partition containing all of the data on the virtual
machine.

In more detail

Computers typically segment hard drives into three
components.

Partition Tables
Partitions
File Systems

A partition table tells the disk where the beginning and
end of a partition exists and usually the file system.

The Partitions are simple blocks of storage.

The File System tells the operating system how to
access the partition.

A normal installation of Windows for example, will look
like:

Drive Block 0

Partition Table:

Starts at Block 256 File System NTFS

Drive Block 256

Partition

background image

Data

LVM

Linux has LVM handle all the tough parts of managing
the positions of data blocks, and makes everything else
easier for the user.

Hence it is called the Logical Volume Manager, because
it manages logical volumes.

What this means is when we create a LVM Partition, we
are actually using an LVM "File System", which looks
like this:

Drive Block 0

Partition Table:

Starts at Block 256 File System LVM

Drive Block 256

Partition

Data

We can then create partitions using LVM which
automatically handles the location of the data inside the
LVM Partition.

For example, we can create two LVM Partitions for a
Windows and Linux Virtual Machine, which would look
like this:

Drive Block 0

Partition Table:

Starts at Block 256 File System LVM

Drive Block 256

Partition

Data

LV Windows

background image

LV Linux

Only an operating system that recognizes LVM can see
"LV Windows" or "LV Linux" though, so if you plug that
drive into a basic Windows machine, it won't be able to
do anything with the data.

'''Finally we have Xen'''

With Xen we give one of those partitions to our virtual
machine, but as stated it sees that as a hard drive.

If we use the previous drive configuration and just add a
Windows Virtual Machine to

LV Windows

our

configuration looks like this:

Drive Block 0

Partition Table:

Starts at Block 256 File System LVM

Drive Block 256

Partition

Data

LV Windows

LV Block 0

Partition Table

Partition at Block
256 File System
NTFS

LV Block 256

Partition

Data

LV Linux

At this point, we cannot simple access the data on our
Xen partition because it has been sub-partitioned.

If you try simply using

mount

it will fail, because it

requires the offset to the start of the NTFS sub-partition!

background image

Using mount With LVM Virtual

Partitions

To get the offset of the partition, we will use a tool like

fsdisk

or

parted

:

Using

unit B

we indicate bytes which will be used by

the mount command, and we get this:

Important to note that Windows 7 creates a 100MB Boot
Partition, so the first item with the boot flag is actually the
boot partition, we want the

Start

value of the second,

which is

105906176B

.

We can now mount the partition using:

That is how we can access the data inside our sub-
partitions.

Obviously it is ill-advised to access the data while the
operating system is running, mounted disks will NOT
inform the running platform of changes, so data can get
overwritten and cause all kinds of problems.

dd

The utility

dd

is probably one of my favorites among the

sudo parted /dev/mapper/xen-windows unit B print

Number Start End Size Type File system Flags
1 1048576B 105906175B 104857600B primary ntfs boot
2 105906176B 171796594687B 171690688512B primary ntfs

sudo mount -o offset=105906176 /dev/mapper/xen-windows /path/to/mount/dir

background image

tools that come with most linux/unix systems.

It can be used to create an exact copy of data, and is not
subject to a number of limitations with higher level
software.

It is not the fastest utility, but supplemented with tools
like gzip you can compress a complete backup into a
size similar to that you might see with professional
software.

Here is a backup Example that compressed an 80GB
Windows LV with 70GB of Data into a 43GB GZipped

Image File:

Alternative one-liner (Takes a lot longer since it has to
compress "on the fly"):

I can also use this command to restore my system from
that compressed image:

Once again, one-liner alternative:

There are plenty of other uses I have found for the

dd

command, but backups and restoration make Linux
amazingly easy to restore after terrible failure, and
believe me in writing this guide I encountered that plenty
of times.

sudo dd if=/dev/mapper/xen-windows of=/path/to/backup/windows.img bs=1M
sudo gzip -9 windows.img

sudo dd if=/dev/mapper/xen-windows bs=1M | sudo gzip -9 &gt; /path/to/backup/windows.img.gz

sudo gzip -d /path/to/backup/windows.img.gz
sudo dd if=/path/to/backup/windows.img of=/dev/mapper/xen-windows bs=1M

sudo gzip -dc /path/to/backup/windows.img.gz | sudo dd of=/dev/mapper/xen-windows bs=1M

background image

You've reached the end of my guide!

Thanks for taking your time to read through my
documentation, if you found it at all helpful please feel

free to shoot me an email on the Xen Users email list.

Any feedback or corrections are also welcomed.

I am still working on the PFSense segment and will have
that up as soon as I can.

Personal Configuration

Data

This section contains documentation on my own
personal configurations and is not fit for third-party
consumption.

Configurations for these components will vary by user
and system.

SSH Configuration

For my personal SSH configuration, I like to change the
default port from 22 to something different. This helps
with security and eliminates conflicting port-forwards for
external mapping.

To do this we simply need to open up

/etc/ssh/sshd_config

and change the 22 in

port 22

to another number.

After that you need to restart

ssh

with

sudo service ssh

restart

.

background image

Samba Configuration

My Samba configuration consists of three 1TB Hard

drives setup with LVM and multiple partitions for an
organized file storage system.

Samba configuration consists of:

Preparing mount points for each partitions
Setting up automatic drive mounting at boot time
Adding a samba user and changing permissions
on said mount points

As of this point, Samba should already be installed, and
a group called

sambashare

should exist on the system.

Our first objective will be adding our user to that group,
and adding our user to the samba users with:

sudo usermod -aG sambashare cdelorme
sudo smbpasswd -a cdelorme

When prompted for a password, this is the new
password you will be able to use to access shares over
the network, and can be different than your login
password.

Note: We must logout and back in before the system
recognizes us as part of the

sambashare

group.

Next we need to prepare a mount point where all of our
shared folders can sit, and by default the system uses

/media

so I will create

/media/samba

:

cd /media
sudo mkdir samba

Currently I have two volume groups named

media

and

background image

data

, and a set of volume groups for each, my goal will

be to create a set of (optionally matching) directories for
each as a mounting point:

/dev/mapper/data-backups
/dev/mapper/data-docs
/dev/mapper/data-software
/dev/mapper/media-games
/dev/mapper/media-multi
/dev/mapper/media-music

So next I run the following command to create
appropriate folders:

Now we need to think about mounting the partitions, but
we need this to happen automatically at boot time, so we

turn to the

/etc/fstab

file.

This file is how we can add new partitions to mount at
boot time, and we have a decision to make.

Drives can be mounted by UUID or root path (eg.

/dev/mapper/...

). I go with UUID, each approach has

its own pros and cons.

To get the UUID of each of my partitions I use this
command:

sudo blkid /dev/mapper/VolumeGroup-VolumeName

Note: No UUID will be provided if a partition has not
been given a file system.

Here is what my

/etc/fstab

file contains:

cd /media/samba
sudo mkdir backups docs software games multi music

# /etc/fstab: static file system information.
#

background image

For simplicity I reboot the system instead of running half
a dozen mount commands, which also verifies my fstab
file entries are valid.

Now for one last step we just need to set permissions on
our share folders and files:

sudo chown -R cdelorme:sambashare /media/samba/

Next we can finally configure Samba itself to share our

# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
proc /proc proc defaults 0 0

# Originals
#/dev/mapper/xen-dom0 / ext4 defaults,noatime,discard,errors=remount-ro 0 1
#/dev/mapper/xen-home /home ext4 defaults,noatime,discard,errors=remount-ro 0 2
#/dev/mapper/xen-swap /swap swap sw 0 0

# Revised UUID Listing
UUID=da46053a-c733-4505-ab87-3f510b08e338 /boot ext4 defaults,noatime,discard,errors=remount-ro 0 2
UUID=AB5A-DB78 /boot/efi vfat utf8 0 0
UUID=dfcc1ff9-8f21-44d0-b224-447c66de7dc0 / ext4 defaults,noatime,discard,errors=remount-ro 0 1
UUID=15833a6d-488b-46ac-adfc-e1db4929d096 /home ext4 defaults,noatime,discard,errors=remount-ro 0 2
UUID=d1e67556-cf2e-4166-81b2-b9c8da10af06 none swap sw 0 0

# Samba Shared LVM Drives
UUID=0bbfb482-ca39-43c5-a1f3-39749c9dadaa /media/samba/backups ext4 defaults,errors=remount-ro 0 0
UUID=30097f44-5978-4fc4-b5cf-663dca259363 /media/samba/docs ext4 defaults,errors=remount-ro 0 0
UUID=4fe0435a-7041-402e-abb3-708e258d39cd /media/samba/software ext4 defaults,errors=remount-ro 0 0
UUID=6c612d46-f0b9-405f-8a62-f77bdc266237 /media/samba/music ext4 defaults,errors=remount-ro 0 0
UUID=3f9f7128-d004-4b1a-81d7-e752631127a1 /media/samba/multi ext4 defaults,errors=remount-ro 0 0
UUID=b56f7d04-6037-4bfe-854e-5f7ee9a7da4a /media/samba/games ext4 defaults,errors=remount-ro 0 0

# DVDRom
/dev/sr0 /media/cdrom0 udf,iso9660 user,noauto 0 0

background image

folders, which is a whole different ballgame of its own.

My documentation is scarce, because the default samba
configuration file (located at

/etc/samba/smb.conf

) had

terrible default settings that shared my entire home

folder, so I had to gut it, and after much online reading
and a bit of testing I ended up with this simplified
version:

#======================= Global Settings =======================

[global]
workgroup = WORKGROUP
server string = %h server
dns proxy = no
log file = /var/log/samba/log.%m
max log size = 1000
syslog = 0
panic action = /usr/share/samba/panic-action %d
encrypt passwords = true
passdb backend = tdbsam
obey pam restrictions = yes
unix password sync = yes
passwd program = /usr/bin/passwd %u
passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
pam password change = yes
map to guest = bad user
usershare allow guests = yes

#======================= Share Definitions =======================

[Backups]
create mask = 660
directory mask = 770
path = /media/samba/backups
write list = cdelorme
[Docs]
create mask = 660
directory mask = 770
path = /media/samba/docs
write list = cdelorme
[Software]
create mask = 660
directory mask = 770
path = /media/samba/software

background image

Note: For write and read lists, to specify a group, prefix
with

@

, otherwise use a username. The create and

directory masks are just defaults I found in many
examples.

After that file is saved I simply restart samba with

sudo

service samba restart

and voila everytime the system

boots it will automatically share my folders, which can
now be accessed over the network by any operating
system and with excellent speeds.

I have measured speeds in excess of 100MB over the
network from Xen virtual machines, without interruptions
to other running network services.

References

Samba Config
Input Reference
Keyboard Reference

read list = @sambashare
write list = cdelorme
[Music]
create mask = 660
directory mask = 770
path = /media/samba/music
write list = @sambashare
[Games]
create mask = 660
directory mask = 770
path = /media/samba/games
read list = @sambashare
write list = cdelorme
[Multi]
create mask = 660
directory mask = 770
path = /media/samba/multi
read list = @sambashare
write list = cdelorme

background image

Mount SFTP in File Manager
Get Aptitude Package History

End


Wyszukiwarka

Podobne podstrony:
Comprehensive Debian (poziom)
instalacja debiana
cs175 176 compressed
ŚPIEWNIK HARCERSKI 2 compressed(4)
INSTALACJA DEBIANA, pakiety i kodeki.debLINUX
r00-0-spr-spr, ## Documents ##, Debian GNU Linux
Akumulator do AMA COMPRESSOR ST180H 1 2
Comprendre
Akumulator do HOLMAN COMPRESSORS??1?1S?1SS
Dowody na istnienie Boga i życia po śmierci compressed
Instalacja Debiana2 2 GNU
MIESIĘCZNY KALENDARZ PRACY WEWNĘTRZNEJ, wzrastania duchowego compressed(1)
Śpiewnik Zuchowy compressed(4)
r00-0-spr, ## Documents ##, Debian GNU Linux
JADLOSPIS SO compressed
Debian PowerPC pl
Pizza bez pszenicy compressed(1)
images compressed(1)

więcej podobnych podstron