=======================================
freebsd migratable vnet jail with ipfw
=======================================

This vnet jail will be created with the ability to migrate
off onto hardware or a vm or even another jail in the future.

The host machine will not have a public network.
The jail will have a public network protected using ipfw.


=======================================
HOST promiscuous mode on
---------------------------------------

If you are creating a jail inside of a virtual machine you will need to
turn promiscuous mode on for network bridging to work correctly.
For virtualbox modify the real host hardware nic.

vboxmanage modifyvm metro --nictype1 virtio --nic1 bridged --bridgeadapter1 igb0 --nicpromisc1 allow-all

For vmware locate the vmx file and add an entry for the interface.

ethernet0.noPromisc = "FALSE"


=======================================
HOST create a migratable zfs structure
---------------------------------------

The host will have a dedicated zfs zpool named tank.

zpool create -O compression=zstd -O checksum=sha512 -O atime=on -o autoexpand=off -m /export tank /dev/ada1

.......................................

If you need a fresh start you can unmount and destroy anything that exists. Careful!

zfs list -o name,mounted -r tank/jail | grep 'yes' | cat -n - | sort -rn | awk '{ print "zfs umount -f " $2 }'

zfs destroy -fR tank/jail

.......................................

zfs create                                                                   tank/jail

zfs create                                                                   tank/jail/judo
zfs create                                                   -o canmount=off tank/jail/judo/root
zfs create                                                                   tank/jail/judo/disk

zfs create -o mountpoint=/export/jail/judo/root/zroot                        tank/jail/judo/disk/zroot
zfs create -o mountpoint=none                                                tank/jail/judo/disk/zroot/ROOT
zfs create -o mountpoint=/export/jail/judo/root                              tank/jail/judo/disk/zroot/ROOT/default
zfs create -o mountpoint=/export/jail/judo/root/tmp -o exec=on -o setuid=off tank/jail/judo/disk/zroot/tmp
zfs create -o mountpoint=/export/jail/judo/root/usr          -o canmount=off tank/jail/judo/disk/zroot/usr
zfs create                                                                   tank/jail/judo/disk/zroot/usr/home
zfs create -o mountpoint=/export/jail/judo/root/var          -o canmount=off tank/jail/judo/disk/zroot/var
zfs create                                         -o exec=off -o setuid=off tank/jail/judo/disk/zroot/var/audit
zfs create                                         -o exec=off -o setuid=off tank/jail/judo/disk/zroot/var/crash
zfs create                                         -o exec=off -o setuid=off tank/jail/judo/disk/zroot/var/log
zfs create                                                                   tank/jail/judo/disk/zroot/var/mail
zfs create                                                     -o setuid=off tank/jail/judo/disk/zroot/var/tmp

.......................................

create a separate dataset for data storage

zfs create -o mountpoint=/export/jail/judo/root/export                        tank/jail/judo/disk/tank

.......................................

This query should produce no output indicating a proper zfs structure was created

zfs get -r -t filesystem canmount,mountpoint tank/jail/judo/disk | grep -v -e 'PROPERTY' -e ' inherited' -e ' default' -e ' local'


=======================================
HOST install the jail
---------------------------------------

bsdinstall jail /export/jail/judo/root

.......................................

Or do a zfs recieve into tank/jail/judo/disk/zroot


=======================================
HOST /etc/rc.conf
---------------------------------------

hostname="metro"

# the private interface igb0
ifconfig_igb0="inet 192.168.10.10 netmask 255.255.255.0"
defaultrouter="192.168.10.1"

# the public interface igb1
ifconfig_igb1="up"

# the public interface igb1
# private = igb0 bridge0 epair0x
# public  = igb1 bridge1 epair1xx
cloned_interfaces="\
  bridge0 epair0  \
  bridge1 epair10 \
  "

# private bridge
ifconfig_bridge0="\
  addm igb0 \
  addm epair0a \
  "
ifconfig_epair0a="up"

# public bridge
ifconfig_bridge1="\
  addm igb1 \
  addm epair10a \
  "
ifconfig_epair10a="up"

# enable jails
jail_enable="YES"

# start listed jails on boot otherwise start all
jail_list="judo"

# enable ipfw for jails
firewall_enable="YES"
firewall_type="open"

.......................................

service devfs restart
service netif restart
service routing restart


=======================================
HOST /etc/jail.conf
---------------------------------------

exec.clean;
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
allow.mount;
mount.devfs;
path = "/export/jail/${name}/root";
exec.consolelog = "/var/log/jail_${name}.log";

judo {
    vnet;
    vnet.interface  = "epair0b";
    vnet.interface += "epair10b";
}


=======================================
HOST /export/jail/judo/root/etc/rc.conf
---------------------------------------

hostname="judo"

ifconfig_epair0b="inet 101.101.101.101 netmask 255.255.255.248"
defaultrouter="101.101.101.97"

# NOTE: If the host has ipfw enabled then the jail must also have ipfw enabled
# and type set to "open" as a minimum to allow network traffic to flow
# ie firewall_type="open"
firewall_enable="YES"
firewall_script="/usr/local/bin/ipfw.sh"
firewall_logging="YES"


=======================================
HOST /export/jail/judo/root/usr/local/bin/ipfw.sh
---------------------------------------

#!/bin/sh
#
# ipfw
#
# just update this script when you need any changes and run it

#------------------------------------------------------------------------------
# Setup

# Add rule
rule="ipfw -q add"

# loopback
loop="lo0"

# private LAN interface
lan="epair0b"

# public WAN interface
wan="epair10b"

#------------------------------------------------------------------------------
# Flush out the list first

ipfw -q -f flush

#------------------------------------------------------------------------------
# Allow loopback In/Out

# $rule 1100 allow all  from any to any via $loop
$rule 1100 allow tcp  from any to any via $loop
$rule 1110 allow udp  from any to any via $loop
$rule 1120 allow icmp from any to any via $loop

#------------------------------------------------------------------------------
# Allow LAN In/Out

# $rule 2100 allow all  from any to any via $lan
$rule 2100 allow tcp  from any to any via $lan
$rule 2110 allow udp  from any to any via $lan
$rule 2120 allow icmp from any to any via $lan

#------------------------------------------------------------------------------
# Allow Existing Connections In/Out

# Allow if packet matches an existing entry in the dynamic rules table
$rule 4100 check-state

#------------------------------------------------------------------------------
# Deny WAN Attack in

# Deny non-routable reserved address spaces
$rule 5100 deny all from 192.168.0.0/16  to any in via $wan #RFC 1918 private IP
$rule 5110 deny all from 172.16.0.0/12   to any in via $wan #RFC 1918 private IP
$rule 5120 deny all from 10.0.0.0/8      to any in via $wan #RFC 1918 private IP
$rule 5130 deny all from 127.0.0.0/8     to any in via $wan #loopback
$rule 5140 deny all from 0.0.0.0/8       to any in via $wan #loopback
$rule 5150 deny all from 169.254.0.0/16  to any in via $wan #DHCP auto-config
$rule 5160 deny all from 192.0.2.0/24    to any in via $wan #reserved for docs
$rule 5170 deny all from 204.152.64.0/23 to any in via $wan #Sun cluster interconnect
$rule 5180 deny all from 224.0.0.0/3     to any in via $wan #Class D & E multicast

# Deny public pings
# $rule 5200 deny icmp from any to any in via $wan

# Allow traceroute out
$rule 5200 unreach port udp  from any to any 33434-33524        via $wan
$rule 5210 allow        icmp from any to any icmptypes 0,3,4,11 via $wan

# Deny ident
$rule 5300 deny tcp from any to any 113 in via $wan

# Deny all Netbios services.
$rule 5400 deny tcp from any to any 137 in via $wan
$rule 5410 deny tcp from any to any 138 in via $wan
$rule 5420 deny tcp from any to any 139 in via $wan
$rule 5430 deny tcp from any to any  81 in via $wan

# Deny fragments
$rule 5500 deny all from any to any frag in via $wan

# Deny ACK packets that did not match the dynamic rule table
$rule 5600 deny tcp from any to any established in via $wan

#------------------------------------------------------------------------------
# Allow WAN Out (client)

# all open
$rule 6100 allow tcp  from me to any out via $wan setup keep-state
$rule 6110 allow udp  from me to any out via $wan keep-state
$rule 6120 allow icmp from me to any out via $wan keep-state
# ping
# $rule 6100 allow icmp from any to any     out via $wan keep-state
# whois
# $rule 6110 allow tcp  from any to any  43 out via $wan setup keep-state
# ntp
# $rule 6200 allow udp  from any to any 123 out via $wan keep-state
# ssh
# $rule 6300 allow tcp  from any to any  22 out via $wan setup keep-state
# dns
# $rule 6400 allow tcp  from any to any  53 out via $wan setup keep-state
# $rule 6410 allow udp  from any to any  53 out via $wan keep-state
# http
# $rule 6500 allow tcp  from any to any  80 out via $wan setup keep-state
# https
# $rule 6510 allow tcp  from any to any 443 out via $wan setup keep-state
# smtp
# $rule 6600 allow tcp  from any to any  25 out via $wan setup keep-state

#------------------------------------------------------------------------------
# Allow WAN In (server)

# ssh
# $rule 7100 allow log tcp from any to me  22 in via $wan setup keep-state
# dns master
# $rule 7200 allow udp     from any to me  53 in via $wan keep-state
# dns slave
# $rule 7300 allow tcp     from any to me  53 in via $wan setup keep-state
# http
# $rule 7400 allow tcp     from any to me  80 in via $wan setup keep-state
# https
# $rule 7500 allow tcp     from any to me 443 in via $wan setup keep-state
# smtp
# $rule 7600 allow tcp     from any to me  25 in via $wan setup keep-state
# smtp ssl
# $rule 7610 allow tcp     from any to me 465 in via $wan setup keep-state
# smtp starttls
# $rule 7620 allow tcp     from any to me 587 in via $wan setup keep-state
# pop3
# $rule 7700 allow tcp     from any to me 110 in via $wan setup keep-state
# pop3s
# $rule 7710 allow tcp     from any to me 995 in via $wan setup keep-state
# imap4
# $rule 7800 allow tcp     from any to me 143 in via $wan setup keep-state
# imaps
# $rule 7810 allow tcp     from any to me 993 in via $wan setup keep-state

#------------------------------------------------------------------------------
# Deny & Log

# Deny & Log all other incoming connections
$rule 9100 deny log all from any to any in via $wan

# Deny & Log everything else
$rule 9200 deny log all from any to any


=======================================
HOST /export/jail/judo/root/etc/sysctl.conf
---------------------------------------

# ipfw logging
net.inet.ip.fw.verbose_limit=10


=======================================
HOST start the jail
---------------------------------------

service jail start judo

jls


=======================================
HOST upgrade the jail
---------------------------------------

freebsd-update -j judo fetch

freebsd-update -j judo install

service jail stop judo

jls -d

service jail start judo

.......................................

To upgrade the jail to a new major or minor version you
must first upgrade the host and then upgrade the jail

freebsd-update -j judo -r 13.2-RELEASE upgrade

freebsd-update -j judo/root install

service jail stop judo

jls -d

service jail start judo

freebsd-update -j judo install

service jail stop judo

jls -d

service jail start judo

.......................................

pkg -j judo update

pkg -j judo upgrade

service jail stop judo

jls -d

service jail start judo


=======================================
HOST create jail user account
---------------------------------------

zfs create tank/jail/judo/disk/zroot/usr/home/oeng
jexec judo pw groupadd -g 16401 -n oeng
jexec judo pw useradd -n oeng -c "Owen Eng" -d /home/oeng -g oeng -G wheel -s /bin/sh -u 16401 -m
jexec judo mkdir /home/oeng/.ssh
jexec judo chown -r oeng:oeng /home/oeng
jexec judo chmod -r 0700 /home/oeng


=======================================
HOST delete jail user account
---------------------------------------

jexec judo pw userdel -n oeng
jexec judo pw groupdel -n oeng
zfs destroy -fR tank/jail/judo/disk/zroot/usr/home/oeng


=======================================
HOST delete a jail
---------------------------------------

service jail stop judo
zfs destroy -fR tank/jail/judo

.......................................

remove entries from /etc/jail.conf
remove entries from /etc/rc.conf


=======================================
:0)
=======================================