seasharp
/
anvil
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Local modifications to ClusterLabs/Anvil by Alteeve
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
652 Commits
2 Branches
0 Tags
21 MiB
 
 
 
 
 
 
Tag: Branch: Tree: e8efbab343
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'e8efbab343'
${ noResults }
anvil/notes

704 lines
34 KiB
Raw Blame History

============
DISABLE KSM!
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/chap-ksm
NEXT; -
RHEL 8 package changes:
chrony replaces ntp
cockpit replaces virt-manager (deprecated)
e1000e replaces e1000 driver
tmux replaces screen
iproute2 replaces bridge-utils (See "ip link" and man bridge.)
==] UEFI Setup [======================================================================
ignoredisk --only-use=vdb,vda
clearpart --none --initlabel
part raid.312 --fstype="mdmember" --ondisk=vda --size=19966
part raid.293 --fstype="mdmember" --ondisk=vdb --size=512
part raid.319 --fstype="mdmember" --ondisk=vdb --size=19966
part raid.286 --fstype="mdmember" --ondisk=vda --size=512
raid pv.326 --device=pv1 --fstype="lvmpv" --level=RAID1 raid.312 raid.319
raid /boot/efi --device=efi --fstype="efi" --level=RAID1 --fsoptions="umask=0077,shortname=winnt" --label=efi raid.286 raid.293
volgroup striker_vg0 --pesize=4096 pv.326
logvol swap --fstype="swap" --size=4096 --name=lv_swap --vgname=striker_vg0
logvol / --fstype="xfs" --size=15852 --label="lv_root" --name=lv_root --vgname=striker_vg0
======================================================================================
DOCS; -
- Explanation of 'comps.xml' (package grouping) - https://pagure.io/fedora-comps
- Firewalld
- https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-using-firewalld-on-centos-7
- PXE;
- https://docs.fedoraproject.org/en-US/fedora/f28/install-guide/advanced/Network_based_Installations/
- https://docs.fedoraproject.org/en-US/Fedora/26/html/Installation_Guide/chap-pxe-server-setup.html
- UEFI PXE notes - https://www.syslinux.org/wiki/index.php?title=PXELINUX#UEFI
- How to write a NetworkManager dispatcher script to apply ethtool commands? - https://access.redhat.com/solutions/2841131
- Setup nodes to log to striker? - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-configuring_netconsole
- Pacemaker can be monitored via SNMP - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/high_availability_add-on_reference/s1-snmpandpacemaker-HAAR
- corosync.conf - https://access.redhat.com/articles/3185291
====
Network planning;
10.x.y.z / 255.255.0.0
10.x.y.z / 255.255.0.0
x = Network;
- BCN = 200 + network
ie: BCN1 = 10.201.y.z
BCN2 = 10.202.y.z
- SN = 100 + network
ie: SN1 = 10.101.y.z
SN2 = 10.102.y.z
y = Device Type.
Foudation Pack;
1. Switches
2. PDUs
3. UPSes
4. Strikers
5. Striker IPMI (BCN only)
Anvil! systems;
1st - 10 = Node IP
11 = Node IPMI
2nd - 12 = Node IP
13 = Node IPMI
3rd - 14 = Node IP
15 = Node IPMI
N...
z = Device Sequence
- Foundation pack devices are simple sequence
- Anvils; .1 = node 1, .2 = node 2, .3 = dr
====
RHEL 8 Firewall
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8-beta/html/configuring_and_managing_networking/assembly_using-firewalls_configuring-networking-with-gnome-gui
=============================
Ports we care about
Porto Number Used by Nets Description
TCP 2224 pcsd bcn It is crucial to open port 2224 in such a way that pcs from any node can talk to all nodes in the cluster, including itself.
UDP 5404 corosync bcn Required on corosync nodes if corosync is configured for multicast UDP
UDP 5405 corosync bcn Required on all corosync nodes (needed by corosync)
TCP 7788+ drbd sn 1 port per resource
TCP 49152-49215 virsh bcn live migration - migration_port_min and migration_port_max attributes in the /etc/libvirt/qemu.conf
NOTE: DHCP listens to raw sockets and ignores firewalld rules. We need to stop dhcpd directly - https://kb.isc.org/docs/aa-00378
* After all changes;
firewall-cmd --zone=public --add-port=49152-49215/tcp --permanent
firewall-cmd --reload
- Paths
If we want to create services or helpers later, look under - /usr/lib/firewalld/
Core firewalld configs, including defaults zones, etc - /etc/firewalld/
* Zones are meant to deal with dynamic environments and aren't that useful in mostly static server environments
* Use 'firewall-cmd' WITHOUT '--permanent' for things like enabling the VNC port for a server. Use '--permanent' for everything else.
====
Striker as PXE server
====
# Bootloader for BIOS
OS="fedora28"
mkdir /var/lib/tftpboot/
cp $(anvil source)/pxe/tftpboot/bios/* /var/lib/tftpboot/
chmod 755 /var/lib/tftpboot/*
# Bootloader for UEFI
cp $(anvil source)/pxe/tftpboot/uefi/* /var/lib/tftpboot/uefi/
chmod 755 /var/lib/tftpboot/uefi/*
# Copy kernel images for tftpboot downloads
mkdir -p /var/lib/tftpboot/${OS}
cp /lib/modules/$(uname -r)/vmlinuz /var/lib/tftpboot/${OS}/
mkinitrd /var/lib/tftpboot/${OS}/initrd.img $(uname -r)
# Configs from anvil source
rsync -av pxe/tftpboot/pxelinux.cfg/default root@f28-striker01:/var/lib/tftpboot/pxelinux.cfg/
rsync -av pxe/tftpboot/pxelinux/uefi root@f28-striker01:/var/lib/tftpboot/pxelinux/
====
DB stuff;
Dump;
su - postgres -c "pg_dump anvil" > /anvil.out
Drop;
su - postgres -c "dropdb anvil" && su - postgres -c "createdb --owner admin anvil" && su - postgres -c "psql anvil"
Reload the DB;
su - postgres -c "dropdb anvil" && su - postgres -c "createdb --owner admin anvil" && su - postgres -c "psql anvil < /anvil.out" && su - postgres -c "psql anvil"
Changes made using tools such as nmcli do not require a reload but do require the associated interface to be put down and then up again. That can be done by using commands in the following format:
* nmcli dev disconnect interface-name
Followed by:
* nmcli con up interface-name
NOTE: RHEL doesn't support direct-cabled bonds - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/ch-configure_network_bonding
ifcfg-X config Notes - /usr/share/doc/initscripts-*/sysconfig.txt (Look for the sections describing files /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-<interface-name>);
- man 5 nm-settings-ifcfg-rh
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-Using_Channel_Bonding#s3-modules-bonding-directives
- /usr/share/doc/kernel-doc-*/Documentation/networking/bonding.txt
iface
* PREFIXx overrules NETMASKx. Use PREFIXx, not NETMASKx.
* The 'x' suffice for PREFIX, NETMASK, etc start at 0 and must count up by 1 at a time.
* ZONE will be useful for the firewall stuff later.
* ETHTOOL_OPTS is deprecated, replaced by using udev rules
* initscripts interpret PEERDNS=no to mean "never touch resolv.conf". NetworkManager interprets it to say "never add automatic (DHCP, PPP, VPN, etc.) nameservers to resolv.conf".
Bond
* resend_igmp & num_unsol_na={1~255} may help if a switch is slow to notice traffic has moved to the new interface. default is 1. Each update is send 200ms apart.
* Bridged interfaces should use BRIDGE_UUID="", _not_ BRIDGE="". The former causes the later to be ignored and the later is only used for possible compatibility reasons.
Bridge
* STP=no is default, we'll test 'yes'.
* DOMAIN="<client_domain>"
=======
virt-manager stores information in dconf-editor -> /org/virt-manager/virt-manager/connections ($HOME/.config/dconf/user)
==== dconf read /org/virt-manager/virt-manager/connections/uris
['qemu+ssh://root@localhost/system', 'qemu+ssh://root@wp-a01n02.remote/system', 'qemu+ssh://root@an-nas02.kw01.alteeve.ca/system', 'qemu+ssh://root@hb-a01n01.remote/system', 'qemu+ssh://root@hb-a01n02.remote/system', 'qemu:///system']
==== dconf read /org/virt-manager/virt-manager/connections/autoconnect
['qemu+ssh://root@localhost/system']
====
# Web - TODO: Setup to auto-use "Let's Encrypt", but make sure we have an offline fall-back
# SELinux
restorecon -rv /var/www
=============================================================
* Network;
** {bc,if,s}nX_{link,bond,bridge}Y naming
** firewall; - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/high_availability_add-on_reference/s1-firewalls-haar
firewall-cmd --permanent --add-service=high-availability
firewall-cmd --add-service=high-availability
firewall-cmd --reload
* Cluster Config;
==== Both nodes
echo Initial1 | passwd hacluster --stdin
systemctl start pcsd.service
systemctl enable pcsd.service
systemctl disable libvirtd.service
systemctl stop libvirtd.service
==== One node
pcs host auth el8-a01n01 el8-a01n02 -u hacluster -p "secret"
### VMs
pcs cluster setup m3-anvil-01 el8-a01n01 el8-a01n02
pcs cluster start --all
pcs stonith create virsh_node1 fence_virsh pcmk_host_list="el8-a01n01" ipaddr="192.168.122.1" passwd="secret" login="root" delay="15" port="el8-a01n01" op monitor interval="60"
pcs stonith create virsh_node2 fence_virsh pcmk_host_list="el8-a01n02" ipaddr="192.168.122.1" passwd="secret" login="root" port="el8-a01n02" op monitor interval="60"
### Real iron.
pcs stonith create ipmilan_node1 fence_ipmilan pcmk_host_list="mk-a02n01" ipaddr="10.201.13.1" password="another secret p" username="admin" delay="15" op monitor interval="60"
pcs stonith level add 1 mk-a02n01 ipmilan_node1
pcs stonith create ipmilan_node2 fence_ipmilan pcmk_host_list="mk-a02n02" ipaddr="10.201.13.2" password="another secret p" username="admin" op monitor interval="60"
pcs stonith level add 1 mk-a02n02 ipmilan_node2
pcs stonith create apc_snmp_node1_psu1 fence_apc_snmp pcmk_host_list="mk-a02n01" pcmk_off_action="reboot" ip="10.201.2.3" port="3" power_wait="5" op monitor interval="60"
pcs stonith create apc_snmp_node1_psu2 fence_apc_snmp pcmk_host_list="mk-a02n01" pcmk_off_action="reboot" ip="10.201.2.4" port="3" power_wait="5" op monitor interval="60"
pcs stonith level add 2 mk-a02n01 apc_snmp_node1_psu1,apc_snmp_node1_psu2
pcs stonith create apc_snmp_node2_psu1 fence_apc_snmp pcmk_host_list="mk-a02n02" pcmk_off_action="reboot" ip="10.201.2.3" port="4" power_wait="5" op monitor interval="60"
pcs stonith create apc_snmp_node2_psu2 fence_apc_snmp pcmk_host_list="mk-a02n02" pcmk_off_action="reboot" ip="10.201.2.4" port="4" power_wait="5" op monitor interval="60"
pcs stonith level add 2 mk-a02n02 apc_snmp_node2_psu1,apc_snmp_node2_psu2
pcs stonith create delay_node1 fence_delay pcmk_host_list="mk-a02n01" wait="60" op monitor interval="60"
pcs stonith level add 3 mk-a02n01 delay_node1
pcs stonith create delay_node2 fence_delay pcmk_host_list="mk-a02n02" wait="60" op monitor interval="60"
pcs stonith level add 3 mk-a02n02 delay_node2
# Either case
pcs property set stonith-max-attempts=INFINITY
pcs property set stonith-enabled=true
### TODO: Look into 'priority-fencing-delay'
# Create a new server resource, stopped, create the location constraint (higher == preferred), then start.
pcs resource create srv01-test ocf:alteeve:server name="srv01-test" meta allow-migrate="true" target-role="stopped" op monitor interval="60" start timeout="INFINITY" on-fail="block" stop timeout="INFINITY" on-fail="block" migrate_to timeout="INFINITY"
pcs constraint location srv01-test prefers mk-a02n01=200 mk-a02n02=100
pcs resource enable srv01-test
- or -
pcs resource update srv01-test ocf:alteeve:server name="srv01-test" meta allow-migrate="true" target-role="stopped" op monitor interval="60" start timeout="INFINITY" on-fail="block" stop timeout="INFINITY" on-fail="block" migrate_to timeout="INFINITY"
# Test
stonith_admin --fence el8-a01n02 --verbose; crm_error $?
pcs resource create srv01-test ocf:alteeve:server name="srv01-test" meta allow-migrate="true" target-role="started" op monitor interval="60" start timeout="INFINITY" on-fail="block" stop timeout="INFINITY" on-fail="block" migrate_to timeout="INFINITY"
pcs constraint location srv01-test prefers el8-a01n01=200 el8-a01n02=100
stonith-max-attempts=INFINITY
cluster-recheck-interval puts an upper bound on the "i give up" time
==== DRBD notes
* resources can contain an US-ASCII character, except for spaces
* A resource is a single replication stream for 1 or more resources, max 65.535 vols per resource
* DRBD does, however, ship with an LVM integration facility that automates the creation of LVM snapshots immediately before synchronization. This ensures that a consistent copy of the data is always available on the peer, even while synchronization is running. See Using automated LVM snapshots during DRBD synchronization for details on using this facility.
** https://docs.linbit.com/docs/users-guide-9.0/#s-lvm-snapshots
* Checksum-based synchronization computes a block's hash on source and target and skips if matching, possibly making resync much faster for blocks rewritten with the same data, but at the cost of CPU. Make this a user-configurable option under the advanced tab.
* Suspended replication allows congested replication links to suspend replication, leaving the peer in a consistent state, but allowing the primary to "pull ahead". When the congestion passes, the delta resyncs. Make this a user-configurable option with scary warnings.
* Online verification can (should?) be run periodically on the server host (verification source will overwrite deltas on the verification target). Perhaps schedule to run once/month? Do resource sequentially as this places a CPU load on the nodes.
* Replication traffic integrity checking uses a given available kernel crypto to verify data integrity on transmission to the peer. If the replicated block can not be verified against the digest, the connection is dropped and immediately re-established; because of the bitmap the typical result is a retransmission.
** Make an option in the advanced tab. Test to see overhead this adds. Choose the lowest overhead algo (within reason)
* Support for disk flushes might be something we want to disable, as it seems to force write-through even with a function FBWC/BBU. Need to test.
* Note; "Inconsistent" is almost always useless. "Consistent" and "Outdated" are able to be used safely, just without whatever happened on the peer after.
* Truck based replication, also known as disk shipping, is a means of preseeding a remote site with data to be replicated, by physically shipping storage media to the remote site.
* Make sure that selinux doesn't block DRBD comms over the SN
* See "5.15.1. Growing on-line" for growing a DRBD resource
** Shrinking online is ONLY possible if the metadata is external. Worth creating *_md LVs? Offline requires backing up and restoring the MD
Provisioning a server will need to:
* Create the LVs
* Open up the DRBD ports
* Create the DRBD resource(s); Find the lowest free rX.res, create it locally and on the peer (if up),
firewall-cmd --zone=public --permanent --add-port=7788-7790/tcp
firewall-cmd --reload
* Provision the server via virt-install
* push the new XML to striker such that the peer's anvil daemon picks it up and writes it out.
[root@el8-a01n01 drbd.d]# drbdsetup status r0 --verbose --statistics
r0 node-id:1 role:Primary suspended:no
write-ordering:flush
volume:0 minor:0 disk:UpToDate quorum:yes
size:10485404 read:9682852 written:0 al-writes:0 bm-writes:0 upper-pending:0 lower-pending:0 al-suspended:no blocked:no
el8-a01n02.alteeve.com node-id:0 connection:Connected role:Secondary congested:no
volume:0 replication:SyncSource peer-disk:Inconsistent done:92.29 resync-suspended:no
received:0 sent:9679140 out-of-sync:808144 pending:6 unacked:3
[root@el8-a01n02 ~]# cat /sys/kernel/debug/drbd/resources/r0/connections/el8-a01n01.alteeve.com/0/proc_drbd
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
ns:24360 nr:10485404 dw:10485404 dr:25420 al:0 bm:0 lo:0 pe:[0;0] ua:0 ap:[0;0] ep:1 wo:2 oos:10461044
[>....................] sync'ed: 0.3% (10212/10236)M
finish: 0:50:01 speed: 3,480 (5,020 -- 3,480) K/sec
99% sector pos: 20970808/20970808
resync: used:0/61 hits:557 misses:2 starving:0 locked:0 changed:1
act_log: used:0/1237 hits:0 misses:0 starving:0 locked:0 changed:0
blocked on activity log: 0
[root@el8-a01n02 ~]# drbdadm primary r0
r0: State change failed: (-1) Multiple primaries not allowed by config
Command 'drbdsetup primary r0' terminated with exit code 11
[root@el8-a01n02 ~]# drbdadm net-options --allow-two-primaries=yes r0
[root@el8-a01n02 ~]# drbdadm net-options --allow-two-primaries=no r0
drbdsetup show all
drbdsetup show all --show-defaults
== virt-install stuff
* Get a list of --os-variants: 'osinfo-query os'
* virt-install --print-xml (or --transient)
* Migate;
# For all resources under the server;
#drbdadm net-options r0 --allow-two-primaries=yes
drbdsetup net-options srv01-c7_0 2 --_name=m3-a02n01.alteeve.com --csums-alg=md5 --data-integrity-alg=md5 --after-sb-0pri=discard-zero-changes --after-sb-1pri=discard-secondary --after-sb-2pri=disconnect --protocol=C --fencing=resource-and-stonith --allow-two-primaries=yes
drbdsetup net-options srv01-c7_0 2 --_name=m3-a02n01.alteeve.com --csums-alg=md5 --data-integrity-alg=md5 --after-sb-0pri=discard-zero-changes --after-sb-1pri=discard-secondary --after-sb-2pri=disconnect --protocol=C --fencing=resource-and-stonith --allow-two-primaries=yes
# Migrate:
virsh -c qemu+ssh://root@m3-a02n02.alteeve.com/system list
virsh migrate --unsafe --undefinesource --live srv01-c7 qemu+ssh://m3-a02n01.alteeve.com/system
virsh -c qemu+ssh://root@m3-a02n02.alteeve.com/system migrate --undefinesource --live srv01-c7 qemu+ssh://m3-a02n01.alteeve.com/system
# Again for all resource under the server;
drbdadm net-options r0 --allow-two-primaries=no
drbdsetup net-options <resource> <target_node_id> --_name=<target_node_name> --allow-two-primaries=yes
virsh migrate --undefinesource --live <server> qemu+ssh://<target_node>/system
drbdsetup net-options <resource> <target_node_id> --_name=<target_node_name> --allow-two-primaries=no
pcs constraint list --full
Location Constraints:
Resource: srv01-c7
Enabled on: m3-a02n02.alteeve.com (score:50) (id:location-srv01-c7-m3-a02n02.alteeve.com-50)
pcs constraint remove location-srv01-c7-m3-a02n02.alteeve.com-50
Set to 90% of BCN bandwidth
migrate-setspeed domain bandwidth
Set the maximum migration bandwidth (in MiB/s) for a domain which is being migrated to another host. bandwidth is interpreted as an
unsigned long long value. Specifying a negative value results in an essentially unlimited value being provided to the hypervisor. The
hypervisor can choose whether to reject the value or convert it to the maximum value allowed.
migrate-getspeed domain
Get the maximum migration bandwidth (in MiB/s) for a domain.
# Provision servers
mkdir /mnt/anvil/{provision,files,archive,definitions}
== Resource Agent; https://github.com/ClusterLabs/resource-agents/blob/master/doc/dev-guides/ra-dev-guide.asc
* A resource agent receives all configuration information about the resource it manages via environment variables. The names of these environment variables are always the name of the resource parameter, prefixed with OCF_RESKEY_. For example, if the resource has an ip parameter set to 192.168.1.1, then the resource agent will have access to an environment variable OCF_RESKEY_ip holding that value.
===
When stopping a server;
14:03 < lge> "on-fail: block"
14:03 < lge> is per operation type.
14:08 < lge> anyways, you can also "on-fail: retry"
OK, set the stop timeout to 60, set 'on-fail: block" and set the failure-timeout to 60 and see how pacemaker reacts.
failure-timeout
===
Migrate servers;
- Let ScanCore set 'node-health' attribute (http://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Explained/index.html#s-node-health)
- Set 'migration-limit' to '1' to enforce serial live migration (http://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/1.1/html-single/Pacemaker_Explained/index.html#s-cluster-options).
Migrate a single server by setting a location constraint against the node we want the VM off of.
- If anything goes wrong, the server will enter a blocked state in pacemaker.
- Recovery needs to be 'unmanage -> clean' to avoid a stop call.
11:57 <@kgaillot> for your design, that sounds right. between cleanup and manage, i'd make sure there was a PE run without any pending actions blocked by the unmanaging -- you can either look at the logs on the DC, run "crm_simulate -SL", or just check the status for a bit
11:58 <@kgaillot> you can play around with it by putting a higher preference on the to-be-cleaned node, to make sure it *does* move when you re-manage. that way you can see what logs/simulate/status look like
12:07 <@kgaillot> i'm thinking if you do crm_resource --reprobe instead of cleanup in the above sequence, that should prevent anything unexpected
12:07 <@kgaillot> unmanage -> adjust preferences if needed -> reprobe resource -> wait for probe results to come back in, and if status looks good -> re-manage
12:08 <@kgaillot> the reprobe will wipe the entire resource history and fail counts for the resource, causing pacemaker to recheck the current status on all nodes. if the status then shows the resource running where you expect/want it, with no errors, then it's not going to do anything further
12:09 <@kgaillot> (in 2.0, cleanup only erases the history where the resource has failed, while reprobe erases the history regardless)
12:13 <@kgaillot> if there are no failures in the resource history, there should be no risk of a full stop. if there is no resource history at all, then after reprobe, there should be no risk of any actions (assuming you've set up location preferences and stickiness how you want them)
Recover from a failed migration;
reset location to prefer current host -> unmanage resource -> cleanup resource -> manage resource
(running on node 2, so re-add location constraint - basically, make sure location constraint favours current host)
https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#chapter-1-introduction
==== Sample kickstart for Fedora28 netinstall
#version=DEVEL
ignoredisk --only-use=vda
# Partition clearing information
clearpart --none --initlabel
# Use graphical install
graphical
# Use network installation
url --url="http://10.1.4.1/rhel8/x86_64/iso/"
# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_CA.UTF-8
# Network information
network --bootproto=dhcp --device=ens3 --ipv6=auto --activate
network --hostname=localhost.localdomain
# Root password
rootpw --iscrypted $6$fyAht.3wBVlRGgqG$5dqIv2NrBD87uA51fxuoic/t2G93pXPUjVlh27Avg20ZGY409SK8cMVgABswF.krJSVIyoHfIChXNfpP/qTjI1
# Run the Setup Agent on first boot
firstboot --enable
# Do not configure the X Window System
skipx
# System services
services --enabled="chronyd"
# System timezone
timezone Etc/GMT --isUtc
# System bootloader configuration
bootloader --location=mbr --boot-drive=vda
%packages
@^server-product-environment
%end
%addon com_redhat_kdump --disable --reserve-mb='128'
%end
%anaconda
pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty
pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok
pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty
%end
====
==== M3 Striker Kickstart
# Setion 1
# Command section – Refer to Chapter 2 for a list of kickstart options. You must include the required options.
### NOTE: The %packages, %pre, %pre-install, %post, %onerror, and %traceback sections are all required to be closed with %end
# Section 2
# The %packages section – Refer to Chapter 3 for details.
# Section 3:
# The %pre, %pre-install, %post, %onerror, and %traceback sections – These sections can be in any order and are not required. Refer to Chapter 4, Chapter 5, and Chapter 6 for details.
====
0 root@pulsar:/var/lib/libvirt/images# dev_PATH=$(udevadm info /dev/sdb | grep -e ID_PATH=)
0 root@pulsar:/var/lib/libvirt/images# if [[ $dev_PATH == *"usb"* ]]; then echo "USB drive"; elif [[ $dev_PATH == *"nvme"* ]]; then echo "NVMe drive"; elif [[ $dev_PATH == *"ata"* ]]; then echo "SATA drive"; elif [[ $dev_PATH == *"scsi"* ]]; then echo "SCSI drive"; fi
USB drive
0 root@pulsar:/var/lib/libvirt/images# dev_PATH=$(udevadm info /dev/sda | grep -e ID_PATH=)
0 root@pulsar:/var/lib/libvirt/images# if [[ $dev_PATH == *"usb"* ]]; then echo "USB drive"; elif [[ $dev_PATH == *"nvme"* ]]; then echo "NVMe drive"; elif [[ $dev_PATH == *"ata"* ]]; then echo "SATA drive"; elif [[ $dev_PATH == *"scsi"* ]]; then echo "SCSI drive"; fi
SATA drive
0 root@pulsar:/var/lib/libvirt/images# dev_PATH=$(udevadm info /dev/nvme0n1 | grep -e ID_PATH=)
0 root@pulsar:/var/lib/libvirt/images# if [[ $dev_PATH == *"usb"* ]]; then echo "USB drive"; elif [[ $dev_PATH == *"nvme"* ]]; then echo "NVMe drive"; elif [[ $dev_PATH == *"ata"* ]]; then echo "SATA drive"; elif [[ $dev_PATH == *"scsi"* ]]; then echo "SCSI drive"; fi
NVMe drive
[root@localhost ~]# dev_PATH=$(udevadm info /dev/sda | grep -e ID_PATH=)
[root@localhost ~]# if [[ $dev_PATH == *"usb"* ]]; then echo "USB drive"; elif [[ $dev_PATH == *"nvme"* ]]; then echo "NVMe drive"; elif [[ $dev_PATH == *"ata"* ]]; then echo "SATA drive"; elif [[ $dev_PATH == *"scsi"* ]]; then echo "SCSI drive"; fi
SCSI drive
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sdb/device/model
Flash Disk
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sda/device/model
SanDisk SDSSDXPS
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/nvme0n1/device/model
INTEL SSDPEKKW512G7
### Stuff only NVMe has
1 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/nvme0n1/device/subsysnqn
nqn.2014.08.org.nvmexpress:80868086BTPY63650FPG512F INTEL SSDPEKKW512G7
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sda/device/subsysnqn
cat: /sys/class/block/sda/device/subsysnqn: No such file or directory
1 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sdb/device/subsysnqn
cat: /sys/class/block/sdb/device/subsysnqn: No such file or directory
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/nvme0n1/device/serial
BTPY63650FPG512F
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sda/device/serial
cat: /sys/class/block/sda/device/serial: No such file or directory
1 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sdb/device/serial
cat: /sys/class/block/sdb/device/serial: No such file or directory
1 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/nvme0n1/device/transport
pcie
0 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sda/device/transport
cat: /sys/class/block/sda/device/transport: No such file or directory
1 root@pulsar:/var/lib/libvirt/images# cat /sys/class/block/sdb/device/transport
cat: /sys/class/block/sdb/device/transport: No such file or directory
Disk size:
/sys/class/block/sda/size * <block size>
=====
man pages
1 Executable programs or shell commands
5 File formats and conventions eg /etc/passwd
7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
8 System administration commands (usually only for root)
A manual page consists of several sections.
Conventional section names include NAME, SYNOPSIS, CONFIGURATION, DESCRIPTION, OPTIONS, EXIT STATUS, RETURN VALUE, ERRORS, ENVIRONMENT, FILES, VERSIONS, CONFORMING TO, NOTES, BUGS, EXAMPLE, AUTHORS, and SEE ALSO.
The following conventions apply to the SYNOPSIS section and can be used as a guide in other sections.
bold text type exactly as shown.
italic text replace with appropriate argument.
[-abc] any or all arguments within [ ] are optional.
-a|-b options delimited by | cannot be used together.
argument ... argument is repeatable.
[expression] ... entire expression within [ ] is repeatable.
====
=======================================
chrissie's cluster script
# Run these commands on all nodes:
cp ../ifup-local /sbin
/sbin/ifup-local
pcs host auth -uhacluster -phacluster amy.chrissie.net anna.chrissie.net clara.chrissie.net fanny.chrissie.net
if [ "$(hostname)" != "amy.chrissie.net" -a "$(hostname)" != "amy" ]
then
exit
fi
# and these on just one:
pcs cluster setup taroxVMs amy.chrissie.net anna.chrissie.net clara.chrissie.net fanny.chrissie.net
pcs cluster start --all
sleep 30
pcs stonith create fence-virsh fence_virsh ipaddr=192.168.100.1 login=root passwd=christine pcmk_host_map="amy:rhel8-1;anna:rhel8-2;clara:rhel8-3;fanny:rhel8-4"
if [ ! -e '/dev/an-a01n01_vg0/srv09-psql_0' ];
then
/sbin/lvcreate -L 69GiB -n srv09-psql_0 an-a01n01_vg0
fi
virt-install --connect qemu:///system \
--name srv09-psql \
--ram 4096 \
--arch x86_64 \
--vcpus 2 \
--cpu Nehalem,+fsgsbase \
--cdrom '/shared/files/Win2016_Server_64-bit_English.iso' \
--boot menu=on \
--disk path='/shared/files/virtio-win.iso',device=cdrom --force\
--os-variant win2k8 \
--network bridge=ifn_bridge1,model=virtio \
--disk path=/dev/an-a01n01_vg0/srv09-psql_0,bus=virtio,cache=writethrough \
--graphics spice \
--noautoconsole --wait -1 > /var/log/anvil-server_srv09-psql.log &
# Migration;
pcs constraint remove $(pcs constraint show --full | grep ban-srv07-el6 | perl -pe 's/^.*?id:(.*?)\)/$1/')
# Attach a network interface:
virsh attach-interface win2019_test bridge ifn_bridge1 --live --model virtio
# Detach a network interface:
virsh detach-interface win2019_test bridge --mac 52:54:00:ee:b5:1d
# Change the MTU of a device;
ip link set <dev> mtu 9000
# Change the MTU of an interface in windows (not tested yet - http://networking.nitecruzr.net/2007/11/setting-mtu-in-windows-vista.html)
netsh interface ipv4 set subinterface "Local Area Connection" mtu=nnnn store=persistent
yum install kernel-2.6.32-754.27.1.el6.x86_64 kernel-devel-2.6.32-754.27.1.el6.x86_64 kernel-headers-2.6.32-754.27.1.el6.x86_64
DRBD Notes;
* c-max-rate is default 100M. Detect when the SN is 10Gbps and up to 750M
* drbdsetup show <resource> --show-defaults <- shows defaults
When a node comes online, and the peer and DR Host are both UpToDate, manually run the commands so that the
restarting node connects to the DR first, so that it will sync from DR and minimize IO load on the peer node.
The sequence of commands needed to up a resource are exposed by 'drbdadm up -d <res>'. Ie:
[root@el8-a01n02 ~]# drbdadm -d up srv02-hi
drbdsetup new-resource srv02-hi 1 --auto-promote=yes
drbdsetup new-minor srv02-hi 2 0
drbdsetup new-peer srv02-hi 0 --_name=el8-a01n01 --allow-two-primaries=no --after-sb-0pri=discard-zero-changes --after-sb-1pri=discard-secondary --after-sb-2pri=disconnect --timeout=100 --protocol=C --fencing=resource-and-stonith
drbdsetup new-peer srv02-hi 2 --_name=el8-a01dr01 --allow-two-primaries=no --after-sb-0pri=discard-zero-changes --after-sb-1pri=discard-secondary --after-sb-2pri=disconnect --timeout=100 --protocol=A --fencing=dont-care
drbdsetup new-path srv02-hi 0 ipv4:10.101.10.2:7791 ipv4:10.101.10.1:7791
drbdsetup new-path srv02-hi 2 ipv4:10.101.10.2:7793 ipv4:10.101.10.3:7793
drbdmeta 2 v09 /dev/node_8da3d2fe/srv02-hi_0 internal apply-al
drbdsetup attach 2 /dev/node_8da3d2fe/srv02-hi_0 /dev/node_8da3d2fe/srv02-hi_0 internal --disk-flushes=no --md-flushes=no
drbdsetup connect srv02-hi 0
drbdsetup connect srv02-hi 2
Manual split brain recovery;
# Both nodes;
drbdadm disconnect <res>
# Node to discard;
drbdadm connect --discard-my-data <res>
# Node to save data on;
drbdadm connect <res>
==================
# Server srv01-sql, example showing two disks in one VM.
resource srv01-sql {
on mk-a02n01 {
node-id 0;
volume 0 {
device /dev/drbd_srv01-sql_0 minor 0;
disk /dev/rhel/srv01-sql_0;
meta-disk internal;
}
volume 1 {
device /dev/drbd_srv01-sql_1 minor 1;
disk /dev/rhel/srv01-sql_1;
meta-disk internal;
}
}
on mk-a02n02 {
node-id 1;
volume 0 {
device /dev/drbd_srv01-sql_0 minor 0;
disk /dev/rhel/srv01-sql_0;
meta-disk internal;
}
volume 1 {
device /dev/drbd_srv01-sql_1 minor 1;
disk /dev/rhel/srv01-sql_1;
meta-disk internal;
}
}
on mk-a02dr01 {
node-id 2;
volume 0 {
device /dev/drbd_srv01-sql_0 minor 0;
disk /dev/rhel_new-dr/srv01-sql_0;
meta-disk internal;
}
volume 1 {
device /dev/drbd_srv01-sql_1 minor 1;
disk /dev/rhel_new-dr/srv01-sql_1;
meta-disk internal;
}
}
### NOTE: Remember to open the appropriate firewall port!
# firewall-cmd --zone=SN1 --permanent --add-port=7788/tcp --permanent
# firewall-cmd --zone=SN1 --permanent --add-port=7788/tcp
connection {
host mk-a02n01 address 10.101.12.1:7788;
host mk-a02n02 address 10.101.12.2:7788;
net {
protocol C;
fencing resource-and-stonith;
}
}
connection {
host mk-a02n01 address 10.101.12.1:7789;
host mk-a02dr01 address 10.101.12.3:7789;
net {
protocol A;
fencing dont-care;
}
}
connection {
host mk-a02n02 address 10.101.12.2:7790;
host mk-a02dr01 address 10.101.12.3:7790;
net {
protocol A;
fencing dont-care;
}
}
}
==================