Difference between revisions of "Diskless netboot"

 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
[[Category:Linux]]
 
[[Category:Linux]]
 
 
 
=Context and aim=
 
 
 
 
==Aim==
 
 
I want to achieve the following configuration:
 
 
[[File:Smartcards network.png|none|Target network diskless configuration]]
 
 
 
Key points:
 
 
* Each client is a diskless station.
 
 
* I want to use the same distribution everywhere.
 
 
 
 
For all this tutorial I'll be using a local network 172.16.50.0/24 with 172.16.50.2 as master server.
 
 
 
 
 
'''Diskless''' station means:
 
 
The clients don't need any hard drive to run, they will retrieve the file system from the TFTP server and use a NFS share as hard drive. The system will only run in RAM disk. So:
 
 
* All the clients will share the installation, configuration files and so on.
 
 
* Each client will run a dedicated instance of the operating system in his own RAM disk
 
 
* Logs will be centralized on the common NFS server - so we don't loose data on each reboot.
 
 
* The user will be able to choose the O.S to run on boot - thanks to a PXE menu
 
 
 
 
 
Monitoring point of view:
 
* Each client will have a local '''SNMP agent''' + a dedicated '''zabbix agent''' to send back its status. In my case, as I'm using Java, I'll install the Oracle JDK and I will use some '''JMX''' beans too.
 
* The current server ''smartcard-gw'' will also have a '''SNMP agent''' + a dedicated '''zabbix agent''' so I can monitor it too.
 
 
 
File access:
 
* The common NFS share will be exposed by a Samba server as a network file-share. Therefore I'll be able to browse for it and access it easily.
 
 
 
 
 
==Requirements==
 
 
To achieve that you need to have:
 
* [[DHCP server]]
 
* [[NetBoot server]]
 
 
 
Optional:
 
* [[DNS server]]
 
 
  
  
Line 167: Line 104:
  
 
By default the ''deboostrap'' Ubuntu 14.04 LTS image is 239 Mo. With the applications we're gonna use that size will increase to about 1 or '''1.3 Go''' depending if you copy (or not) the kernel sources. It may even take 1.6 Go if you're using XFCE frontend.
 
By default the ''deboostrap'' Ubuntu 14.04 LTS image is 239 Mo. With the applications we're gonna use that size will increase to about 1 or '''1.3 Go''' depending if you copy (or not) the kernel sources. It may even take 1.6 Go if you're using XFCE frontend.
 
 
 
 
 
=Installation=
 
 
 
'''NFS support'''
 
<syntaxhighlight lang="bash">
 
apt-get install nfs-kernel-server nfs-common
 
</syntaxhighlight>
 
 
'''Debootstrap (manage netboot image)'''
 
<syntaxhighlight lang="bash">
 
apt-get install debootstrap
 
</syntaxhighlight>
 
 
 
'''Initramfs (to manage "virtual disks")'''
 
<syntaxhighlight lang="bash">
 
apt-get install initramfs-tools
 
</syntaxhighlight>
 
 
 
 
 
 
=Preparation=
 
 
You have to create a dedicated folder on your server where you will host the distributions kernels + Boot settings.
 
 
<syntaxhighlight lang="bash">
 
mkdir -p /tftpboot/pxelinux.cfg
 
chmod -R 755 /tftpboot/pxelinux.cfg
 
 
mkdir -p /tftpboot/images
 
chmod -R 755 /tftpboot/images
 
</syntaxhighlight>
 
 
 
The ''pxelinux.cfg/'' folder is mandatory. Inside you can provide:
 
* configuration for a specific IP @ or hostname
 
* configuration for a group
 
* default configuration (required)
 
 
 
 
 
 
=Boot menu and Kernel setup=
 
 
The first thing to do is to setup a booting kernel. To do so we'll use the "syslinux" files.
 
 
 
'''Reminder'''
 
If your client(s) will use some smart-cards driver then you MUST install these drivers on the on the NFS server + reboot the server ; before going through the following steps. See [[Drivers#Smart-card_drivers]]
 
 
 
 
 
==Root file: pxelinux.0==
 
 
The '''pxelinux.0''' is the root file. That's the file that allows the netboot.
 
 
This is the file that is serve by the TFTP server.
 
 
<syntaxhighlight lang="bash">
 
cp /usr/lib/syslinux/pxelinux.0 /tftpboot/
 
</syntaxhighlight>
 
 
 
==Create NetBoot menu | defaults==
 
 
Now, we have to specify which kernel to use and which distributions are available for NetBoot.
 
 
 
Create the default configuration file:
 
 
<syntaxhighlight lang="bash">
 
vim /tftpboot/pxelinux.cfg/default
 
</syntaxhighlight>
 
 
 
Put the following:
 
 
<syntaxhighlight lang="bash">
 
# Debian 7.x
 
LABEL wheezy
 
    kernel images/wheezy/vmlinuz
 
    initrd images/wheezy/initrd.img
 
 
# Ubuntu 14.04
 
LABEL trusty
 
    kernel images/trusty/vmlinuz
 
    initrd images/trusty/initrd.img
 
 
 
# Prompt user for selection
 
PROMPT 1
 
# No timeout
 
TIMEOUT 0
 
</syntaxhighlight>
 
 
* Each LABEL is a specific configuration that will displayed on the NetBoot menu.
 
* PROMPT 0 = enable user prompt so you can choose the configuration
 
* TIMEOUT 0 = timeout (in seconds) before the default option is chosen. 0 == no timeout
 
 
 
 
Note that I used a reference to "trusty/", that's a folder I need to create later on.
 
 
 
 
==Init Kernel files==
 
 
 
===Create directories===
 
 
Create the target kernel folders. You should create 1 folder for each distribution you'd like to provide in NetBoot.
 
 
<syntaxhighlight lang="bash">
 
# Debian 7.x
 
mkdir -p /tftpboot/images/wheezy
 
 
# Ubuntu 14.04
 
mkdir -p /tftpboot/images/trusty
 
</syntaxhighlight>
 
 
 
===Prepare ''initramfs'' to boot over NFS===
 
 
'''This step must to be run on the machine that has the kernel you are going to serve to your clients'''.
 
 
 
>>> In our case it has to be run on the TFTP server
 
 
 
 
Copy initramfs settings for PXE boot
 
 
<syntaxhighlight lang="bash">
 
cp -r /etc/initramfs-tools /etc/initramfs-pxe
 
</syntaxhighlight>
 
 
 
Adjust PXE boot configuration
 
 
<syntaxhighlight lang="bash">
 
cd /etc/initramfs-pxe/
 
vim /etc/initramfs-pxe/initramfs.conf
 
</syntaxhighlight>
 
 
 
 
Add / adjust the following options:
 
 
<syntaxhighlight lang="bash">
 
BOOT=nfs
 
MODULE=netboot
 
</syntaxhighlight>
 
 
 
 
===Copy and prepare kernel===
 
 
 
You have to copy your current kernel files to the boot folder:
 
 
<syntaxhighlight lang="bash">
 
# Debian 7.x
 
cp /boot/vmlinuz-`uname -r` /tftpboot/images/wheezy/vmlinuz
 
cp /boot/initrd.img-`uname -r` /tftpboot/images/wheezy/initrd.img
 
 
# Ubuntu 14.04
 
cp /boot/vmlinuz-`uname -r` /tftpboot/images/trusty/vmlinuz
 
cp /boot/initrd.img-`uname -r` /tftpboot/images/trusty/initrd.img
 
</syntaxhighlight>
 
 
 
 
Enable NFS boot on target kernel:
 
 
<syntaxhighlight lang="bash">
 
mkinitramfs -d /etc/initramfs-pxe -o /tftpboot/images/trusty/initrd.img
 
</syntaxhighlight>
 
 
 
Adjust rights:
 
 
<syntaxhighlight lang="bash">
 
chmod -R 755 /tftpboot/images/
 
</syntaxhighlight>
 
 
 
 
Notes:
 
 
* Do NOT use some symlink for "vmlinuz" and "initrd.img" !! It won't work.
 
 
* If you don't want to use `uname -r` [current kernel version and architecture] then adjust the values to target kernel number + architecture
 
 
* You have to run ''mkinitramfs'' for each kernel you'll provide
 
 
* Don't forget to adjust the rights to 755 for every distribution
 
 
 
 
 
=NFS server setup=
 
 
 
==Configuration==
 
 
The NFS configuration is done in the '''/etc/exports''' file
 
 
<syntaxhighlight lang="bash">
 
vim /etc/exports
 
</syntaxhighlight>
 
 
 
Add something like that:
 
 
<syntaxhighlight lang="bash">
 
### list of available O.S
 
  /nfs/trusty        172.16.50.0/24(ro,no_root_squash,no_subtree_check,async,insecure)
 
  /nfs/wheezy        172.16.50.0/24(ro,no_root_squash,no_subtree_check,async,insecure)
 
### common share
 
  /nfs/common        172.16.50.0/24(rw,no_root_squash,no_subtree_check,async,insecure)
 
</syntaxhighlight>
 
 
 
Adjust "172.16.50.0/24" to your own network address
 
 
* rw : Allow clients to read as well as write access
 
* ro : Read only access
 
* insecure : Tells the NFS server to use unpriveledged ports (ports > 1024).
 
* no_subtree_check : If the entire volume (/users) is exported, disabling this check will speed up transfers.
 
* async : async will speed up transfers.
 
* no_root_squash: This phrase allows root to connect to the designated directory.
 
 
 
- NOTE -
 
 
* It's always a good idea to use Read-Only if you plan to share this disk. That will avoid user to mess with your image!
 
 
* There must not be any space between network IP and "("
 
 
* If you plan to share a NFS to all users - like my '''/nfs/common''' - don't forget to set ''chmod -R 777 /nfs/common''
 
 
 
==Security==
 
 
Like TFTP, this part is insecure !
 
 
You must restrict the access to your NFS server by a firewall script and filtering BEFORE reaching the LAN !
 
 
 
 
NFS is using dynamic ports numbers because it runs over '''rpcbind'''. Making NFS using specifics port is a pain in the ass !! :(
 
 
So, instead of that you should allow your LAN communication.
 
 
 
<syntaxhighlight lang="bash">
 
    IPTABLES=`which iptables`
 
    LAN_ADDRESS="172.16.50.0/24"
 
 
    # Allow LAN communication
 
    $IPTABLES -A INPUT -s $LAN_ADDRESS -d $LAN_ADDRESS -m state ! --state INVALID -j ACCEPT
 
    $IPTABLES -A OUTPUT -s $LAN_ADDRESS -d $LAN_ADDRESS -m state ! --state INVALID -j ACCEPT
 
</syntaxhighlight>
 
 
 
 
==Management==
 
 
<syntaxhighlight lang="bash">
 
service nfs-kernel-server {status|start|stop|restart}
 
</syntaxhighlight>
 
 
 
==Test the server==
 
 
 
Install the NFS v4 client:
 
<syntaxhighlight lang="bash">
 
apt-get install nfs-common
 
</syntaxhighlight>
 
 
 
To mount the default path:
 
<syntaxhighlight lang="bash">
 
mount -t nfs nfs-server:/ /mnt
 
</syntaxhighlight>
 
 
You'll see: "/nfs"
 
 
 
It's better to do:
 
<syntaxhighlight lang="bash">
 
mount -t nfs nfs-server:/nfs /mnt
 
</syntaxhighlight>
 
 
 
 
  
  
Line 516: Line 147:
 
* Automatic [Puppet || Chef] configuration: [[Diskless image configuration - script setup]]
 
* Automatic [Puppet || Chef] configuration: [[Diskless image configuration - script setup]]
  
 
 
==Kernel modules and source==
 
 
-- This is an optional step ; but it can avoid many bugs --
 
 
 
If you're using a local kernel as the default NetBoot kernel, then you need to do copy the ''modules'' + ''kernel source'' to every distribution.
 
 
<syntaxhighlight lang="bash">
 
# Copy kernel modules
 
cp -r /lib/modules/`uname -r` /nfs/trusty/lib/modules
 
 
# Copy kernel sources
 
cp -r /usr/src/linux-headers-`uname -r` /nfs/trusty/usr/src
 
</syntaxhighlight>
 
 
 
Note that '''you have to adjust the /nfs'''/XXX
 
  
  
Line 564: Line 176:
 
* [[PXE interactive menu - single level]]
 
* [[PXE interactive menu - single level]]
 
* [[PXE interactive menu - multi level]]
 
* [[PXE interactive menu - multi level]]
 +
 +
 +
 +
 +
=Local server monitoring=
 +
 +
Install the following services:
 +
* [[SNMP client]]
 +
* [[Zabbix agent setup]]
 +
 +
 +
 +
 +
=Other services=
 +
 +
 +
==File sharing==
 +
 +
If you want to expose the NFS common folder as a file-share, you have to install and configure Samba. See: [[Samba server]]
 +
 +
 +
''Note''
 +
 +
Samba is actually better than NFS for the file-share. You should remove Common from /etc/exports and use a samba share instead.
 +
 +
 +
 +
==Management UI (webmin)==
 +
 +
Since there is a lot of services to manage, it's always convenient to use an UI for it. Check [[Webmin]]
 +
 +
 +
 +
==VPN server==
 +
 +
See [[VPN]]
 +
 +
 +
 +
==Apache2 server==
 +
 +
See [[Apache 2]]
 +
  
  

Latest revision as of 14:37, 21 August 2014


Target folder tree (server side)

This is how we'll setup our files and folders:

# TFTP root
/tftpboot/                                   

###############
# Network bootable image(s) using NFS technology
################       

#### Boot file            
/tftpboot/pxelinux.0                   # Initial boot file - only use to load the PXE NetBoot manager
/tftpboot/{menu.c32 || vesamenu.c32}   # PXE interactive menu managers (text or graphical)
/tftpboot/pxelinux.cfg/                # PXE configuration(s)
/tftpboot/pxelinux.cfg/default         # default PXE configuration

#### Kernel file

/tftpboot/images/      

# Debian 7.x [Wheezy] 
/tftpboot/images/wheezy/   
/tftpboot/images/wheezy/vmlinuz
/tftpboot/images/wheezy/initrd.img

# [X]Ubuntu 14.04 [Trusty] 
/tftpboot/images/trusty/  
/tftpboot/images/trusty/vmlinuz
/tftpboot/images/trusty/initrd.img


#### NFS 
# This is where the runnable will be. Each image will be in a dedicated folder.
/nfs/                    

# Debian 7.x [Wheezy] 
/nfs/wheezy/   

# Ubuntu 14.04 [Trusty] 
/nfs/trusty/


Client overview

Each client must have, at least, 4 Go of RAM.


4 GO RAM configuration

This is how we're gonna populate the client:

NetBoot client RAM overview - 4Go


As you can see, each client will have some space dedicated for swap + some RAMdisk to allow writing in /var, /tmp and /proc.


Configuration of a 4Go RAM disk:

  • No swap
  • Local TMPFS (read/write for /dev, /tmp, ...) : 1 Go
    • /tmp = 512 M
    • /var/tmp = 128 M
    • /var/log = 128 M
    • /var/run = 8 M
    • /var/lock = 8 M
    • /run/shm = 256 M
  • O.S (NFS read only) : all the rest ~ 2.8 Go
  • Common share (NFS read write) : Remote disk


2 Go

Due to budget restriction we might encounter some low memory machines with only 2 Go...


This is how we're gonna populate the client:

NetBoot client RAM overview - 2Go


In case of 2Go RAM then you have to use some tricks:

  • No swap
  • O.S (NFS read only) : ~ 1.2 Go
  • Common share (NFS read write) : Remote disk
  • Local TMPFS (read/write for /dev, /tmp, ...) : all the rest
    • /tmp = 372 M
    • /var/tmp = auto
    • /var/log = 128 M
    • /var/run = auto
    • /var/lock = auto
    • /run/shm = auto


How big is the client image ?

By default the deboostrap Ubuntu 14.04 LTS image is 239 Mo. With the applications we're gonna use that size will increase to about 1 or 1.3 Go depending if you copy (or not) the kernel sources. It may even take 1.6 Go if you're using XFCE frontend.


NFS client image

There are different way to setup a NFS client image.

The main ones are:

  • Manually
    • debootstrap
    • copying the install from your server
    • Manual install on a client, then, when the system is ready, copy everything to the NFS share
  • Using script and software like "Puppet" or "Chef"


Setup client distribution

You have to create one target for each distribution you want to serve:

mkdir -p /nfs/trusty
mkdir -p /nfs/wheezy
mkdir -p /nfs/common


- NOTES -

  • The folder name should match your NetBoot settings. Folder name = a LABEL in the NetBoot config.
  • The folder name should match a Linux (Debian like) distribution name


Configure client distribution



Backup distribution

You can create an archive of your current distribution for later restore / re-use.


Compression

cd /nfs
tar cvpjf trusty.tar.bz2 ./trusty


Restoration

cd /nfs
tar -xvjf trusty.tar.bz2

PXE interactive menu

You can create interactive NetBoot menus, see:



Local server monitoring

Install the following services:



Other services

File sharing

If you want to expose the NFS common folder as a file-share, you have to install and configure Samba. See: Samba server


Note

Samba is actually better than NFS for the file-share. You should remove Common from /etc/exports and use a samba share instead.


Management UI (webmin)

Since there is a lot of services to manage, it's always convenient to use an UI for it. Check Webmin


VPN server

See VPN


Apache2 server

See Apache 2



References

Ubuntu diskless how-to: https://help.ubuntu.com/community/DisklessUbuntuHowto


Mind reference: http://mindref.blogspot.se/2011/03/debian-diskless.html


Super video tutorials:


Nice explanation of PXE process: http://www.linux.com/learn/docs/ldp/497-Diskless-root-NFS-HOWTO