Commit Graph

35 Commits

Author SHA1 Message Date
Digimer
cd87c0f521 * Fixed a bug that caused striker-initialize-host to not compile / run.
Signed-off-by: Digimer <digimer@alteeve.ca>
2021-04-01 11:35:44 -04:00
Digimer
70dc0598f2 * Created Storage->manage_lvm_conf() that checks / updates lvm.conf to add a filter to avoid seeing DRBD devices as LVM components. This is now called from striker-initialize-host and scan-drbd.
Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-31 23:59:19 -04:00
Digimer
ec192d9041 * Fixed a bug where backing up a file on a remote machine returned a failure, if the target backup directory had to be created (even if it was created successfully).
* Fixed a bug in a mini bash command to chmod / chown a directory being created on a remote machine.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-23 00:36:17 -04:00
Digimer
3ed857bacd * Bumped logging for striker-auto-initialize-all debugging.
Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-22 15:22:46 -04:00
Digimer
5536e8ff47 * Updated Cluster->assemble_storage_groups() and Cluster->anvil_name_from_uuid() and ->available_resources() to try to detect the anvil_uuid if not passed in.
* Updated Database->insert_or_update_storage_group_members() to use the host_uuid when trying to find existing members.
* Added the skeleton of a bunch of new json endpoints for the new UI features.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-16 02:40:50 -04:00
Digimer
5e9e7e4dde * Removed debug logging from tools.
Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-10 21:14:56 -05:00
Digimer
54496cbeb0 * Added a check to Database->get_ip_addresses() to check is a hash is set before using it, to help avoid unitialized variable messages.
* Updated Remote->test_access() to not used cached SSH access.
* Updated anvil-configure-host to abort if the host is in a cluster.
* Updated anvil-join-anvil to clean up some variable checks to help avoid unitialized variable messages.
* Updated striker-initialize-host to check if an anvil RPM is installed and, if so, not install the Anvil! repo.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-10 20:35:05 -05:00
Digimer
5db09f565d * Updated anvil-join-anvil to actively call a cluster start once per minute while waiting for initial startup.
* Added a check to striker-initialize-host the see if anvil-X RPM is already installed. If so, it will not install the Alteeve repo, even if it's not found.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-10 14:14:04 -05:00
Digimer
3733220b50 * Updated Log->entry() to prefix log lines with the short 'job-uuid', when the log entry is coming from a program running as a job. This is meant to make it easier to break up what log lines belong to what jobs, if multiple jobs are running at the same time (ie: when initializing multiple nodes / dr hosts in parallel).
* Updated Remote->call() to return ('!!error!!', '!!error!!', 9999) when an error hits. Made Remote->test_access() explicitely check for '1' to be returned in order to confirm access, fixing a bug where bad target value caused false positives. Updated ->_check_known_hosts_for_target() to no longer explicitely check for 'ssh-rsa' so that machine keys using different cyphers are detected as being in known_hosts properly.
* Updated striker-auto-initialize-all to initialize nodes and DR hosts networks before trying to form them into an Anvil!. Fixed several other bugs as well. More testing is needed, but it works now.
* Updated striker-initialize-host to check for the alteeve repo and, it not found, check for accress to alteeve.com. If access, it will install our repo now.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-03-10 02:26:09 -05:00
Digimer
1b65f53faa * Remove host-health from the 'hosts' table as it wasn't needed, given the 'health' table. Bumped the SQL version to 0.0.2
* Updated Get->os_type() to use 'cat' instead of Storage->read_file() because 'rsync' may not be available when it is called during striker-initialize-host calls.
* Updated Database methods to skip 'oui' and 'state' during resync.
* Updatedb striker-initialize-host to detect when it's initializing a CentOS Stream Node / DR Host and enable the HA repo.
* Created the tools/striker-auto-initialize-all tool, which is very much incomplete, that will allow for the rapid creation of a full Anvil! from freshly installed machines autonomously.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-02-22 19:22:47 -05:00
Digimer
1a520b03d5 * Cleaned up a lot of logging in anvil-daemon and tools it calls.
* Deleted anvil-jobs as it never ended up being used.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-02-08 13:39:34 -05:00
Digimer
cda51e562d * Finished porting scan-hpacucli, the last M2 scan agent!
* Updated Database->insert_or_update_temperature() to accespt a 'delete' parameter (which, surprise, deletes a record).
* Updated (but not yet tested) the RPM .spec to require that 'core' and the other three packages are required to be the same version.
* Updated scan-ipmitool and scan-storcli scan agents to now delete temperature data belonging to lost sensors.
* Fixed tools/striker-manage-install-target to add multiple missing packages.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-11-25 19:06:21 -05:00
Digimer
7e6e345513 * Updated tools/striker-manage-install-target to now check to see if the striker is a RHEL host. If not, the packages in the RHEL High Availability add-on are merged into the main package list. If it is RHEL though, a search is made for nodes that it can access and once one is found that is a matching RHEL version / arch, and has Internet access, it is used as a proxy to download the packages in the HA add on and then pulls those packages to the local repo.
* Updated Get->os_type() to work on local and remote hosts.
* Fixed a but in tools/striker-initialize-host where calls to Network->find_matches() where being checked properly for success/failure.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-11-13 03:14:36 -05:00
Digimer
0f7267eae1 * Moved the '_host_name', '_short_host_name', and '_domain_name' private methods in Tools.pm over to Get.pm (removing the leading '_' in the method names).
* Created 'Cluster->which_node' that returns 'node1' or 'node2' to indicate which node a host is.
* Continued working on scan_cluster; decided to make it not host-dependent.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-09-20 00:27:36 -04:00
Digimer
82acb4e104 * Fixed a resync bug where bridges needed to sync before bonds
* Re-enabled user-selected BCN subnet ranges (needs more testing).

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-09-03 22:52:38 -04:00
Madison Kelly
30f2b3fa8e * Switched all hash 'local' keys to be the host's short user name. Untested, likely bugs to be fixed in the next commit.
Signed-off-by: Madison Kelly <mkelly@alteeve.ca>
2020-08-18 19:34:08 -04:00
Digimer
99afd2e936 * Fixed a bug in Database->manage_anvil_conf() where initializing a host set the DB information with the wrong DB port and password.
* Fixed a bug in Get->host_type() the type wasn't being set for nodes and dr hosts.
* Fixed a bug in Validate->host_name() where the wrong method was being called.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-07-17 22:49:25 -04:00
Digimer
de43ea3ac1 * Renamed all Validate->is_X to Validate->X. Also created Validate->ipv6() to validate IPv6 addresses using Data::Validate::IP (and added it as a requirement to the .spec base RPM).
* Added the fix from the last commit for System->call to handle returned data without an ending newline to Remote->call.
* Got more work done on System->update_hosts(). It's able to add new hosts, but misses the short and FQDN host names. Need to fix that and the verify existing / manual entries aren't molested.

Signed-off-by: digimer <digimer@pulsar.alteeve.com>
2020-07-07 01:18:38 -04:00
Digimer
453f5c6223 * Fixed a bug where $anvil->nice_exit() was being passed 'exit' instead of 'exit_code' as a parameter.
* Update striker manifest run to add an entry into the 'anvils' table, and pass the anvil_uuid to the jobs rather than the various host_uuid's.
* Fixed a bug in the 'anvils' SQL procedure that copied data into the history schema (a few columns were missing).
* Updated anvil-configure-host to reboot when finished to be certain network changes have taken effect. Also updated the handling of virsh bridges to delete the autostart symlinks if libvirtd daemon isn't running.
* Added some logic to anvil-daemon to call 'anvil-update-states' with the -v{1,3} flag depending on the active debug level.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-06-24 00:39:56 -04:00
Digimer
76e9352717 * Added a flag that tells anvil-daemon when a node is having it's network mapped. When this happens, open ssh connections are closed each loop and only tasks related to mapping the network run. This improves responsiveness in Striker when reporting which network links have come up or gone down.
* Fixed a bug in Database->insert_or_update_variables() where, if 'update_value_only' was set but not variable_uuid was passed or could be found, an (incomplete) INSERT would be attempted.
* Added support for generating module metadata when setting up local repos on Striker.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-01-09 01:52:04 -05:00
Digimer
530d379f59 * Started work on caching network state change in tools/anvil-update-states.
* Fixed a bug where ip_addresses could break resync when 2+ machines had the same IP (ie: 192.168.122.1).
* Updated logging of DB transactions to show the DB host's IP instead of the UUID.
* Updated Get->date_and_time to take a 'use_utc' parameter to return the time using GMT time instead of the host's TZ.
* Updated anvil-daemon to periodically call tools/anvil-update-states. Also upadted anvil-daemon to delay daily jobs by 2 hours except for the dashboard with the highest sorted UUID to minimize dual runs of tasks that only need to run once per day per cluster.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-04 00:02:19 -05:00
Digimer
0dfd2ddbf0 * Renamed 'ip' to 'ip_address' in Striker->parse_all_status_json() and related functions.
* Got the menu for mapping a host's network displaying (much work still to be done).
* Updated the anvil.js funtion to run dependent on the page being shown. For the main menu, the json is now properly reread and display updated as json content changes.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-29 23:37:11 -05:00
Digimer
628f7faa45 * Updated the RPM spec file to generate '/etc/anvil/type.X' files to directly indicate the machine type. Updated System->get_host_type() to check for these files directly, falling back to parsing the host name if they don't exist.
* Created Database->get_hosts_info() (though it's not at all finished) that will write out a unified JSON file contain all data known about all hosts/Anvil! systems. This will be later used to create the WebUI parts.
* Also created, but also not finished, Network->load_interfces() that will work sort of like ->load_ups, but include all interfaces regardless of if they have an IP or not.
* Fixed a bug where the new bridge_interface_note parameter didn't exist in the Database->insert_or_update_bridge_interfaces() method.
* Updated anvil-update-states() to only write out the JSON/XML files if it's running on a dashboard. For nodes and DR hosts, it just needs to update the database.
* Created a new hook in anvil-daemon that will call tasks on a machine that is configured.
* As per RHEL 8.1 release notes, changed the package 'dnf-utils' to 'yum-utils' in the packages to load for install target repos.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-06 11:26:21 -05:00
Digimer
b436500a54 * Started working on getting database archiving working. It's close to done, but needs testing/debugging and other finery.
* Created Database->get_host_from_uuid() that takes a host UUID and returns the host's name.
* Reworked Network->find_matches() to return both the match's IP and subnet.
* Finished getting tools/striker-initialize-host to add all known peers to the target, using IPs on the target's available subnet.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 01:53:52 -04:00
Digimer
b1a175baa0 * Finished Network->load_ips().
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-23 11:10:07 -04:00
Digimer
7cdd2f60e9 * Created Network->download() to handle downloading a file on the local system. Created ->bridge_info() to parse 'bridge' output. Created ->load_ips() to load IP address information from the database (as opposed to ->get_ips() which queries a system).
* Created Database->insert_or_update_bridge_interfaces() to handle updating the new bridge_interfaces database table that records which network interfaces are connected to which bridges.
* Added 'bridge_mac' and 'bridge_mtu' to the 'bridges' table.
* Started Server->map_network which will, eventually, try to map MAC addresses to IPs and record server -> vnetX -> bridge data. Made getting a server status target-dependent.
* Worked on anvil-update-states to parse and record bridge data.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-12 00:32:32 -04:00
Digimer
183d2d9cce * Updated Storage->backup to append a short UUID to the timestamp to prevent issues if the same file is backed up twice in the same clock second. Also fixed a bug with remote_user parameter not having a default.
* Finished the detection of and handling of initialization of a host when the host has no Internet access.
* Disabled (for now) anvil-daemon's check_ssh_keys function.
* Fixed a couple small bugs elsewhere.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-04 22:36:39 -04:00
Digimer
65a27ba2f8 * Created Network->check_internet() that, well, checks to see if a machine has access to the Internet or not.
* Moved System->is_local to Network->is_local, and System->ping to Network->ping.
* Added a check to tools/striker-get-peer-data that will report if the target has Internet access or not.
* Cleaned up the form that prompts the user to enter their Red Hat credentials.
* Updated tools/anvil-manage-keys (and related code) to no longer distinguish by user. If a target is flagged as changed, it is removed from the root and all user's known_hosts files.
* Updated Storage->write_file() and ->update_file() to accept the 'backup' parameter to control if an file that exists is backed up before being updated/replaced.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-04 00:40:30 -04:00
Digimer
3a86bed694 * Fixed tools/striker-initialize-host so that it set the hostname on the target, not locally.
* Updated System->host_name to work locally and on remote targets.
* Renamed all 'hostname' instances to 'host_name' to standardize on a spelling throughout the program.
* Removed use of and dependency on 'hostname'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-02 15:39:21 -04:00
Digimer
b57546e663 * Fixed a bug in Database->manage_anvil_conf() where the wrong password and port was being set.
* Fixed a bug in Get->host_uuid() where the call to get the host UUID from dmidecode was broken.
* Updated striker -> Initialize host to allow the user to set the host name of a node or host being initialized, allowing it to be registered with Red Hat under the proper name and make it easier to track which machine is which during initial Anvil! build.
* Fixed a few minor bugs with variable insertions into translated strings.
* Updated striker-initialize-host to use a progressive progress value rather than statically assigned steps.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-02 02:04:22 -04:00
Digimer
3e01537209 * Updated how bad keys are removed. No longer are speciifc lines removed by number, but now all entries for the given target in the known_hosts file is removed.
* Updated the pxe.txt file to now write a caller for anvil-update-issue in /etc/NetworkManager/dispatcher.d/ifup-local to have the /etc/issue file is updated as soon as the network is brought up, before the GDM login prompt is shown.
* Fixed a couple bugs in tools/anvil-manage-keys, including to ensure that the permissions are retained when a file is updated.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-27 23:21:24 -04:00
Digimer
2ca19699b8 * Added abunch of required packages to tools/striker-manage-install-target to support installing offline nodes, DR hosts and Striker dashboards.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-24 11:39:09 -04:00
Digimer
b9a0cc4d56 * Finished the initial tools/striker-initialize-host!
* Created Tools->refresh to reload anvil.conf in one call.
* Created Anvil::Tools::Network to hold network-related tasks.
** Created Network->is_remote() that tests to see if a string (containing a target) refers to the remote machine (versus a local machine). Updated all previous checks to use this new method.
** Moved Get->network_details() and Get->network() to the new Network module. Renamed Get->network() to Network->get_network().
** Made Network->get_ips() work locally and remotely.
** Created Network->find_matches() that compares two scanned machines IPs (via two previous calls to Network->get_ips())
* Created Database->manage_anvil_conf() that will add, update or remove a given database connection in a local or remote anvil.conf file.
* Fixed bugs in Storage->backup() where the bash calls were quite broken. I'm not sure how it ever worked before... x_x
* Updated anvil-daemon to not initialize a database unless it's running on dashboard. Also added a check at the startup of anvil-daemon where it will go into a loop waiting for a database to become available, re-reading anvil.conf each loop.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-22 23:36:59 -04:00
Digimer
a54e0d4e22 * Made a fair bit more progress on tools/striker-manage-install-target. It now registers a system with Red Hat and attaches appropriate subscriptions, updates the OS and (tries) to install anvil-{node,dr}.
* Fixed a bug in Remote->call where a shell call that ended in a newline would work, but throw an error and not get the return code.
* Created Database->get_job_details() which takes a job_uuid and returns the job details, if found.
* Fixed a bug in Jobs->update_progress() where 'clear' wasn't removing the old job_progress data.
* Added the parameters 'no_files' to skip stat'ing/recording non-directories, and 'search_for' which will set the parent directory in 'scan::searched' and stop scanning if found. This allows this method to act as a directory tree scanner and as a search engine.
* Created Striker->get_local_repo() that builds a repo file body suitable for adding to peers, nodes and DR hosts.
* Fixed bugs in the Striker WebUI related to initializing a target node / DR host.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-18 18:48:09 -04:00
Digimer
badaa39b7a * Got the node/dr host initialization form to the point where it can test access and decide if it should show the Red Hat account form. Decided that for M3, node/dr host setup will now be a four-stage process; initial install (over PXE), initialization (install the proper anvil-{node,dr} RPM and connect to the database), setup/map the network, and then add to an Anvil! pair.
* Updated striker to no longer try to SSH to a remote machine. To enable this, we'd have to give apache a shell and an SSH key, which is dumb and dangerous when considered.
* Created tools/striker-get-peer-data which is meant to be invoked as the 'admin' user (via a setuid c-wrapper). It collects basic data about a target machine and reports what it finds on STDOUT. It gets the password for the target via the database.
* Updated anvil-daemon to check/create/update setuid c-wrapper(s), which for now is limited to call_striker-initialize-host.
* Created Anvil/Tools/Striker.pm to store Striker web-specific methods, including get_peer_data() which calls tools/striker-initialize-host via the setuid admin call_striker-initialize-host c-wrapper.
* In order to allow striker via apache to read a peer's anvil.version, which it can no longer do over SSH, any connection to a peer where the anvil.version is read is cached as /etc/anvil/anvil.<peer>.version. When Get->anvil_version is called as 'apache', this file is read instead.
* Updated Database->resync_databases() and ->_find_behind_databases() to ignore the 'states' table.
* Created tools/striker-initialize-host which will be called as a job to initialize a node/dr host.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-16 00:17:02 -04:00