Commit Graph

39 Commits

Author SHA1 Message Date
Digimer
bbad058b33 * Created a new tool, anvil-watch-bonds, which is a live monitor of bonds and interfaces designed to be run from the command line on a given host.
* Created Words->center_text that takes a string (or string key) and centers it to a given string length, padding white spaces on either side of the string as needed.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-05-26 20:24:05 -04:00
Digimer
f833c311ba * To address issues with scancore debugging, we needed a tool to purge old anvils and hosts from the database. The 'test.pl' in this commit contains the new logic that will be merged into tools/striker-purge-host shortly.
* Created Database->find_host_uuid_columns() and ->_find_column() to create a list of tables and column names in the proper order to allow deletion of foreign keys to that deeply nested primary keys can be deleted. Specifically, this was meant for hosts -> host_uuid and anvils -> anvil_uuid, though it should work for other tables.
* Updated html/jquery-ui-1.12.1/package.json to address CVE-2020-7729
* Fixed a bug in the temperature table's history procedure where temperature_weight wasn't being copied.
* Updated anvil-provision-server to support '--anvil' that can take either the anvil-uuid or anvil-name.
* Updated anvil-safe-stop to default the stop-reason to 'user'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-05-08 02:02:46 -04:00
Digimer
ac6531ddf2 * Fixed a bug where Words->load_agent_strings() wouldn't process strings without new-lines in them (caused in the last fix).
Signed-off-by: Digimer <digimer@alteeve.ca>
2021-02-03 18:53:56 -05:00
Digimer
3f04c9031b * Fixed a bug in Words->parse_banged_string() Where the flattened string wasn't being used for the variable substitutions.
* This resolves issue #23.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-02-01 12:27:05 -05:00
Digimer
8127c70237 * Support for OS type selection was missing in tools/anvil-provision-server, this commit adds support for this, as well as some infrastructure to support it. This includes a new 'sys::servers::os_short_list' variable that contains a CSV of main OSes to show in a "short" list (the full list is massive). This variable can be set by the user in anvil.conf. Also added job progress calls that were missing through the storage config.
* Created a new tools/striker-parse-os-list tool that parses 'osinfo-query os' and prints out entries for words.xml for any new OSes.

Signed-off-by: Digimer <digimer@alteeve.ca>
2021-01-16 03:30:51 -05:00
Digimer
4f33eeef2e * This commit introduces a new concept called "Storage Groups". Given that LVs back DRBD resources in M3, there needed to be a way to determine which VGs would be used when creating the backing DRBD resources, and how large those LVs could be (based on the minimum free space of the VGs in a group). A new, as yet incomplete Get->available_resources() method will handle determining what resources are available to grow exist
ing or create new servers.
* Created Database->insert_or_update_storage_groups() and ->insert_or_update_storage_group_members() to manage the new associated tables. Together, these tables create storage groups and track the VGs that are members of the group.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-12-28 01:22:21 -05:00
Digimer
1d03a386d3 * Created Database->get_bridges() that, surprise, loads data from the 'bridges' table.
* Started work on Get->available_resources() that will take an 'anvil_uuid' and figure out what resources are still available for use by new servers or that can be added to existing servers.
* Fixed a bug in ScanCore->agent_startup() where tables weren't being generated properly from the agent's SQL file.
* Made Storage->change_mode() return silently if it's called without a mode being passed. This happens frequently and is harmless so it's not worth filling the logs with errors.
* Renamed the 'start_time' key to 'at_start' when recording files' MD5 sums in Storage->record_md5sums and ->check_md5sums.
* When we moved the directory scan logic out of the 'scancore' daemon and into 'Storage->scan_directory', the logic to record scan agent names in 'scancore::agent::<file>' was removed. This broke a few things and, so, it was restored when it was found that a file starts with 'scan-' and the directory matches the scancore agent directory.
* Moved the 'scancore' daemon's 'load_agent_strings' to 'Words'
* Updated Words->parse_banged_string() to look for variables in the format 'value=X:units=Y' and translate it properly.
* Fixed a bug in scan-ipmitool where discovered sensor INSERT SQL queries were queued, but not committed.
* Fixed a bug in scan-storcli where a while loop was broken, preventing execution.
* Fixed a bug in the 'scancore' daemon where it wouldn't exit if sums changed. Fixed a bug where alerts weren't being sent between loops. Fixed a bug where command-line log level wasn't surviving inside the main loop.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-12-23 01:10:23 -05:00
Digimer
44dc4f4b47 * Fixed a bug in Words->string() where not having the parameter 'file' set caused the default 'words.xml' to be specified, preventing strings from scan agents from being used.
* Started work on the new scan-cluster scan agent that will parse out and store data from the pacemaker CIB.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-09-16 19:32:53 -04:00
Digimer
fe7cdb18fb * Updated all methods to add (or fix) logging the method entry.
* More work done on Email->send_email() to, well, actually send email (which it isn't doing yet, but it's close).
* Updated Words->key() to include the bad key name when no entry for the requested key exists in the words.xml file.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-09-06 01:52:03 -04:00
Digimer
911523dfce * Got a lot of work done in generating emails. Doesn't work yet, but the code to generate emails for recipients using their preferred language and alert level is done (though limited testing so far).
* Dropped support for supporting imperial measurements in generated emails.
* Created Database->get_alerts() to read in alert data and ->get_recipients() to get the list of alert recipients.
* SQL Schema changes;
** Added 'alert_processed' to 'alerts' to track what alerts have been processed.
** Changed 'recipient_new_level' to 'recipient_level' now that we're only using 'notifications' as a per-host override for user/hosts alert levels.
** Removed 'recipient_units' as we're no longer supporting non-metric values.
* Updated Alert->register() to take strings for the alert level (which gets translated to integers).
* Created Email->get_current_server() to returned the mail_server_uuid of the active mail server (if any). Created ->send_alerts() to process unprocessed alerts and send emails to recipients.
* Updated Words->parse_banged_string() to take the 'language' parameter.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-09-05 01:18:42 -04:00
Digimer
1498e1b53c * Got server migration working using ocf:alteeve:server in a test environment!
* Converted most 'eval { }' calls to localize $@ and test the output of the eval, instead of checking to see if $@ was set.
* Converted all 'local' hash references to instead use the short host name of the local machine as a new standard.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-08-19 18:54:09 -04:00
Digimer
3c2f25a860 * Added 'fence_delay' fence agent to handle the corner cases where an IPMI BMC had crashed until a power cycle, and PDU fencing was effected, but failed to report as such.
* Updated Cluster->parse_cib() to take a CIB as a parameter.
* Fixed a bug in Database->get_hosts() where loading the host_ipmi value was filtered through Log->is_secure.
* Updated Striker->get_fence_data() to parse the switches to make it easier to map a fence agent's command line switches to STDIN arguments.
* Created System->parse_arguments() that converts a series of command line switches and their values into a hash. It's similar to Get->switches(), but works on any string.
* Continued work on anvil-join-anvil's fence configuration logic.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-07-30 00:23:47 -04:00
Digimer
ef70f90ba4 * Updated Log->entry() to set the log file handle to UTF-8 when opened.
* Got more work done to System->configure_ipmi() to warm reset HP IPMI BMCs. It also now finds the IPMI user have started the password management.
* Created Words->shorten_string() that shortens a string to a number of bytes (as opposed to shortening to a character length).

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-07-22 22:32:33 -04:00
Digimer
22e7e8b03e * Finished the "run manifest" form / page. Actually saving / pushing the job to nodes is not yet done though.
Signed-off-by: Digimer <digimer@alteeve.ca>
2020-05-26 21:37:02 -04:00
Digimer
e66bc32693 * Added the ability to store, edit and delete UPSes
** Created Database->get_upses() and ->insert_or_update_upses().
** Created Striker->get_ups_data(). This parses the special 'ups_XXXX' strings.
* Updated Validate->is_domain() and added ->is_host_name() to use the Data::Validate::Domain module (which is now required in the core RPM).
* Started work on manifest handling.
* Sorted the language keys alphabetically.

Signed-off-by: Digimer <digimer@alteeve.ca>
2020-05-02 23:05:58 -04:00
Digimer
9e16cdf504 * Created Validate->is_email() to validate email addresses, using Mail::RFC822::Address.
* Finished (but not yet tested) the menu to manage alert recipients.
* Created Words->language_list() that creates a hash reference of available languages.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-31 23:43:52 -07:00
Digimer
0b2439edc3 * Finished the jquery/css stuff for showing the list of hosts that can have their network configured.
* Fixed unclear error logging in Network->find_matches().
* Updated System->generate_state_json() Striker->parse_all_status_json() to determine if another machine can be reached from the local dashboard. If it can be reached, the first matching interface and IP are recorded.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-27 23:26:21 -05:00
Digimer
7e960f1632 * Created the 'oui' database table that stores the parsed OUI data, as processed by the new 'tools/striker-parse-oui' tool. Also created Database->insert_or_update_oui() to handle inserts and updates.
* Fixed a bug in Convert->time() where the suffix was long when it should have been short, and vice-versa.
* Updated Network->download() to check if the target file exists and, if so, to abort unless 'overwrite' is given or the existing file is 0-bytes long. Also updated it to not exit on immediate error after the wget call and instead check to see if a zero-byte file exists and remove it, if so.
* Created Validate->is_hex() to check hexadecimal strings.
* Updated Words->clean_spaces() to remove MS-DOS-style ^M cr/lf characters.
* Updated anvil-daemon to have a section for periodic tasks that run daily, and added striker-parse-oui as well as moved striker-manage-install-target refresh to that check. Also made those tools run on dashboards only.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-16 01:57:59 -04:00
Digimer
3e01537209 * Updated how bad keys are removed. No longer are speciifc lines removed by number, but now all entries for the given target in the known_hosts file is removed.
* Updated the pxe.txt file to now write a caller for anvil-update-issue in /etc/NetworkManager/dispatcher.d/ifup-local to have the /etc/issue file is updated as soon as the network is brought up, before the GDM login prompt is shown.
* Fixed a couple bugs in tools/anvil-manage-keys, including to ensure that the permissions are retained when a file is updated.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-27 23:21:24 -04:00
Digimer
37f36fe99c * Updated kickstart to write the basic tools/anvil-update-issue to a freshly installed machine and run it from cron.
* Updated Remote->call() to detect when a connection fails because the target's known_hosts entry has changed. Still need to add the function to report this to the user.
* Fixed a bug where new-lines in Words->parse_banged_string() where a double-banged word string's variable value would cause an infinite loop.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-24 23:35:31 -04:00
Digimer
db1afb766e * Fixed a bug in Words->parse_banged_string() where a variable being inserted that ended in a new-line would break the variable insertion into the string.
* Switched the icons for prep'ing a node or DR host and building an Anvil!.
* Started work on the node/dr host initial setup webUI.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-05 21:21:03 -04:00
Digimer
324ef351fe * Updated DRBD->get_devices() to properly identify the peer node, when run on an actual node in the cluster (not DR or Striker).
* Created System->active_lv() that, surprise, activates an inactive logical volume. Also created ->check_storage() that parses out the LVM data.
* Fixed a bug in tools/fence_pacemaker that was preventing it from compiling and running.
* Updated ocf:alteeve:server to validate the target server's storage.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-08-06 23:31:35 -04:00
Digimer
b1326e1b4e * Created tools/anvil-download-file which will handle downloading, aborting and reporting status of downloads.
* Started working on Convert->time().
* Changed anvil-manage-files skip /mnt/shared/temp when looking for files to add to the database.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-03-15 02:40:44 -04:00
Digimer
2c23c6beba * Improved infinite loop handling in Log->entry, but broke the Striker UI in the process. To be fixed next...
* Added a 'test' parameter to Log->entry, Storage->make_directory and Words->key to help debug in places that Log->x may not be usable.
* Converted many $anvil->Log->x calls to print if $test to help prevent recursive loops, but not all fixed yet.
* Added the new 'host_keys' database table to the schema for a possible new feature of removing passwords in favour if machines adding peers' public keys to their authorized_hosts file.
* Cleaned up the opening calls to $anvil->Tools->new() in most tools.
* Cleaned up some variables in tools/anvil-update-states after reading their values from files (clean trailing newlines).

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-01-18 03:19:36 -05:00
Digimer
02c4fe1fa1 * Updated all perl module modes to remove the executable bit.
* Updated anvil.sql to add the new tables needed for alert mail delivery.
* Update anvil.sql and Database->initialize to now default the user to 'admin' and swap that out if needed, instead of using the #!variable!user!#' replacement variable.
* Started updating anvil.spec for EL8.
* Added support for 'striker::repo::extra-packages' which users can use to add additional packages to the Striker repositories.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-01-05 18:57:44 -07:00
Digimer
8fad67fc5a * Updated Words->read to default to 'path::words::words.xml' when the 'file' parameter is not passed. Also updated it to check to see if the words file was read before and, if so, clear the data from the previous read before re-reading it.
* Updated anvil-daemon to re-read the main words file on each loop.
* Updated scancore to read and purge each scan agent's words file between invocations.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-12-28 19:06:26 -04:00
Digimer
1c70fceb70 * Fixed a minor bug in Template->get() where a bad template name being passed would call Words->string() without proper arguments and generate error messages.
* Fixed a bad template name in main.html.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-12-27 01:14:09 -04:00
Digimer
f5ae90c941 * Started work on M3 ScanCore!
* Started expanding Alert->register_alert() to actually implement it.
* Improved handling errors in Words->key().
* Started work on Striker's "Anvil!" menu section. Also cleaned up the power handling.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-12-18 15:20:49 -08:00
Digimer
b5dc83d39c * Renamed anvil-configure-striker -> striker-configure-host and anvil-manage-install-target -> striker-manage-install-target as they're both Striker-specific.
* Fixed a bug in Words->parse_banged_string where some variable strings were not being cleared, causing infinite loops.
* Added job progress reporting in striker-manage-install-target, and made it only refresh the RPM repo when '--refresh' is specified (with --force now forcing the issue). This was done to allow adding it into anvil-daemon in such a way that it would only update the RPM repo once a day.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-12-06 04:14:58 -05:00
Digimer
706052105e * Changed Storage->read_file() to default 'force_read' to '1', causing it to always read from disk and ignore cache unless set to '0'.
* After much time wasted chasing a dnf bug (https://bugzilla.redhat.com/show_bug.cgi?id=1641947), tools/anvil-manage-install-target now populates the <DOCROOT>/<os_type>/<os_arch>/os/Packages/ directory with needed RPMs.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-24 02:21:41 -04:00
Digimer
962ff89fc5 * Fixed a bug in Words->parse_banged_string() where values with commas was breaking the processing of the string of variable/value pairs.
* Added '--refresh-json' to anvil-daemon that auto-selects '--run-once', '--main-loop-only' and '--no-start'.
* Updated anvil-update-system to not go more than a second between updates to the progress (save for when we're holding on data from 'dnf').

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-03 02:35:25 -04:00
Digimer
eaca4c885f * Updated tools/anvil-daemon to use the JSON module to build the JSON strings going into jobs.json, instead of doing it manually.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-29 00:33:33 -04:00
Digimer
831ff14d93 * Created Words->parse_banged_string to process strings in the format '<key>[,!!var1!value1!!,!!var2!value2!!,...,!!varN!valueN!!'. Still testing this.
* Made anvil-update-system look for a job_uuid when none is passed.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-28 22:42:46 -04:00
Digimer
19672f8b7c * Started work on the Striker menus, adding mechanisms for "back" and "refresh".
* Fixed a bug where the old style '#!replace!...!#' replacement variables were not being escaped when processing variable insertions into strings.
* Made the body variable be stored in 'form::body' instead of passing around the '$body' variable.
* Created a set of new icons.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-07-04 17:57:25 -04:00
Digimer
9a37f66468 * Decided to abandon using apache security and to integrate it into Striker itself. This will allow for layers of access, easier account sync'ing, etc. Further, there is no proper way to handle logging out on apache, and they use md5 * 1000 iterations (with a salt, to be fair) for security... Not exactly the strongest.
* Created the new Account.pm module for handling account related tasks.
* Moved 'sys::stty' to 'sys::terminal:stty'.
* Created Get->_salt which generates a (pseudo) random salt for internal passwords and Get->_wrap_to that checks the current terminal width.
* Added the parameter 'redirect_stderr' (default '1') to System->call to optionally not redirect STDERR to STDOUT (needed for tput to be useful as STDERR is a terminal of sorts).
* Disabled System->change_apache_password (to be removed entirely later).
* Created Words->_wrap_string to wrap text strings to the active terminal width.
* Added the user's table to the core SQL. Also deleted the empty Anvil/Tools.sql file.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-15 01:55:56 -04:00
Digimer
c21b326f1a * Changed all methods to take a 'debug' argument for setting log level on calls.
* Fixed a bug with resync, but others remain as resync is incomplete (at least for network_interfaces).
* Currently, tools/anvil-update-states is broken while working on the above issue.
* Reworked the jobs table and removed the units/anvil-jobs.service unit. Jobs will be invoked and backgrounded in all calls.
* Started adding missing hidden form fields.
* Updated the 'server' OCF resource agent version and metadata.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-03-07 03:11:55 -05:00
Digimer
d03213b860 * Fixed some string and path issues from the 'anvil' merger.
* Added a 'debug' parameter to System->ping() to allow per-call log levels.

Signed-off-by: Digimer <digimer@alteeve.ca>
2017-12-24 01:49:57 -04:00
Digimer
9648e8ba43 * Create tools/anvil-jobs and units/anvil-jobs.service, which is a new daemon that will handle jobs that can take some time to finish.
* Created Storage->record_md5sums() and Storage->check_md5sums for use in daemons. These will record the md5sums of the program itself, all perl modules and the words file. When check_md5sums is called, it returns '1' if any sums have changed, which daemons can trigger on to exit (and systemd will restart them). Removed the basic md5sum check from anvil-daemon and switched to this.
* Fixed how 'fatalstobrowsers' is invoked so that it only applies to programs running in a browser.

Signed-off-by: Digimer <digimer@alteeve.ca>
2017-12-08 17:04:36 -05:00
Digimer
1cb42080c3 ** Major Changes **
We've decided to give up on trying to keep ScanCore, AN::Tools and Striker as three separate things. We had originally hoped to make ScanCore easily separatable from the Anvil!, but this was adding increasing complexity to the project and complexity is the enemy of reliability.

In this release, AN::Tools becomes Anvil::Tools, all configuration files move to /etc/anvil and all programs and data files move to /usr/sbin/anvil. Words files are now merged, as are SQL schemas (ScanCore agents will still maintain their own, later). The journald tag has changed from 'an-tools' to 'anvil'.

Other changes;
* Tools.t has been updated to handle existing tests. New methods and parameters still need to have tests added though.
* Added a simple test.pl script used for testing things outside the main program. It will be removed before final release.
* Added the simple 'watch_logs' bash script to more easily tail output.

Signed-off-by: Digimer <digimer@alteeve.ca>
2017-10-20 00:19:32 -04:00