Commit Graph

64 Commits

Author SHA1 Message Date
Digimer
02c4fe1fa1 * Updated all perl module modes to remove the executable bit.
* Updated anvil.sql to add the new tables needed for alert mail delivery.
* Update anvil.sql and Database->initialize to now default the user to 'admin' and swap that out if needed, instead of using the #!variable!user!#' replacement variable.
* Started updating anvil.spec for EL8.
* Added support for 'striker::repo::extra-packages' which users can use to add additional packages to the Striker repositories.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-01-05 18:57:44 -07:00
Digimer
4b66379aaa * Added enabling/disabling 'Install Target' feature to Striker's WebUI.
* Fixed a bug in Get->anvil_version where the version of local systems and remote systems differed in closing new lines.
* Fixed a bug in Database->insert_or_update_variables() where the 'debug' parameter wasn't working.
* Renamed System->determine_host_type -> System->get_host_type.
* Fixed a bug in System->get_uptime where there was a newline after the uptime integer.
* Updated anvil-daemon to track and record the state of the Install Target feature on Striker dashboards.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-12-04 04:16:38 -05:00
Digimer
45f290d5e7 * Updated all System->*_daemon methods to not append '.service' to the daemon parameter. Also make all daemon names variables stored in 'sys::daemon::<foo>' and updated existing calls to these methods to use those variables.
* Got anvil-manage-install-target finished creating config files and enabling daemons needed for PXE. Still untested in function though.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-11-01 03:16:07 -04:00
Digimer
07c3b405ad * Starting work on adding "Install Target" function (will likely rename this, but basic same function as IT in m2).
* Added 'sys::database::failed_connection_log_level' to allow silencing of log messages when a Striker peer database is not available.
* Started updating the .spec for the new release to add supported packages needed for PXE/dhcp/tftpboot.
* Added to repo tftpboot files as pulling them out of the packages and moving them into the right place relative to the modest size of adding them directly to our source wasn't justified.
* Created the still very very early 'tools/anvil-manage-firewall' tool.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-10 19:43:23 -04:00
Digimer
9a4cefe3fb * Fixed a bug (which is really more of a work-around for a deeper bug to be addressed later) where the database would keep thinking it needed to resync, even after it had successfully.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-07 02:56:55 -04:00
Digimer
510321d634 * It looks like adding and removing Striker peers (and all the sync'ing stuff behind it) is finally sorted out. Obviously, time will tell for sure, but currently things look good.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-04 20:54:00 -04:00
Digimer
43035ba038 * Fixed a subtle an annoying autovivication bug in Database->write().
* Cleaned up some logging.
* Made the "Reload" buttons work more sensibly and cleaned up some webui display stuff.
* Got deleting peers mostly working (well, it works, but then it goes into a loop thinking it needs to resync the now-gone database until the daemon restarts).
* Fixed a race condition bug where if a job exited between the time that anvil-daemon got a list of PIDs and when it checked to see if that specific pid was alive, a job that actually completed could be restarted.
* Added a loop check to anvil-manage-striker-peers where it would hold until a database connection to the newly added peer was available, preventing a condition where re-adding a peer (and so the host_uuid is in hosts) cause the job belonging to the peer to be recorded locally and then never synced to the peer.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-04 03:37:43 -04:00
Digimer
e79e7fd4f4 * Added 'check_if_configured' to Database->connect(), disabled off, that triggers the check to see if the system is configured or not. Updated anvil-daemon to invoke this at the same time that the md5sums are calculated to see if a reload is needed. This reduces the background system load a fair bit.
* Got more work done on deleting peers from Striker (technically done, but untested so far).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-02 03:31:42 -04:00
Digimer
facefeaccc * Fixed a bug in anvil-daemon where completed jobs could be immediately cleared, causing them to re-run (repeatedly).
* Added 'sys::log_date' which controls if the date and time is pre-pended to log entries.
* Created Get->host_name() which takes a host UUID and returns the 'host_name' from the 'hosts' table, if found.
* Cleaned up some HTML templates and logging.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-28 01:50:38 -04:00
Digimer
90bbcfe9f4 * Updated Database->insert_or_update_jobs() to have 'update_progress_only' conditionally update 'job_status', 'job_picked_up_by', 'job_picked_up_at' and 'job_data'.
* Updated Database methods that can take the caller's file or line to append the local file and line.
* Updated Jobs methods to call Database->insert_or_update_jobs().
* Updated the remaining UPDATEs of 'jobs' table to use 'Database->refresh_timestamp()' for the 'modified_date'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 13:50:48 -04:00
Digimer
bbf194994e * New attempt to solve the resync issue; Created a new method called Database->refresh_timestamp that will update 'sys::database::timestamp', and call it when UPDATE'ing in '->insert_or_update_jobs()'. This should force the 'modified_date' to be unique per update and allow the existing resync approach to work properly. If so, this method will be used later for any instance where a given record can be updated multiple times per run.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 02:15:44 -04:00
Digimer
9bd5dd9a18 Revert to bfc2204.
This reverts commit bfc2204352.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 02:05:07 -04:00
Digimer
950338da6c * More work on Database->_split_query(), though I may be about to revert as this is proving exceedingly complicated and I may have a simpler solution.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 01:43:12 -04:00
Digimer
e07bb0908d * Did more work on parsing queries in Database->_split_query(), but still quite a lot to do.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-24 13:59:21 -04:00
Digimer
a8369170b4 This is the start of a major change!
The resync of the databases was originally designed (on m2) with the expextation that any given column would have only one change per 'modified_date' time. That was never a great approach, but it worked in m2 and just bit me on m3. With job processing, for an example, the job_progress will change repeatedly in one pass, all with the same 'modified_date'. So only one record per run would resync. To fix this, the plan is to drop 'history_id' (and the procedure/trigger in pgsql to copy INSERT and UPDATEs to the history schema). The new plan is to use 'change_uuid' with a per-transaction UUID created in Database so that the per-DB 'history_id' is replaced with a per-update/insert UUID in 'change_uuid'. This will become the unique record used to sync databases, instead or 'modified_date'. To keep things consistent, 'modified_date' was renamed to 'change_date' to match 'change_uuid'. This work is very much "in progress" and not finished.

This commit also changes Get->uuid to use UUID::Tiny to create v4 UUIDs instead of making making a system call to 'uuidgen'. This sped up UUID generation by almost 100x.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-23 16:16:08 -04:00
Digimer
bfc2204352 * Added a row-count check when deciding if a DB resync is needed.
* Updated the Database module to not sort or reorder the 'core_tables' array, and reordered them in the hash they're declared in.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-22 00:54:37 -04:00
Digimer
40aac1d5f6 * Finished adding the 'sessions' database table and associated code.
* Added a check to all 'Database->insert_or_update_*' methods to check if the passed-in reference UUID was found and return an empty string if not.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-21 13:49:28 -04:00
Digimer
e483d6e9a1 * Updated Database->connect() to delete 'database::<uuid>' when a peer's database is not a matching version to avoid later use attempts.
* Fixed a bug in Database->resync_databases() where tables that pluralized with the suffix 'es' were not being singularized properly, so the '_uuid' column wasn't found and preventing resync of that table.
* Fixed a bug in System->maintenance_mode() and System->reboot_needed() where the variable read wasn't restricted to the local host UUID.
* Reordered the config and maintenance mode checks in striker so that configure is shown even if maintenance mode is somehow set without configuration being done.
* Updated the .spec for .16 RPM release.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-14 02:23:33 -04:00
Digimer
c6e781d80b * Adding database peers (local and bidirectional) now works!
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-13 17:08:50 -04:00
Digimer
7bd65f65e5 * Finished the first round of updates to anvil-manage-striker-peers, but the initial resync is failing because of unrelated schema issues.
* Updated Database->insert_or_update_jobs() to also use the job_command when looking for an existing job (when a specific job_uuid was not included).
* Fixed a bug with a missing ? in striker->add_sync_peer function. Also updated it to not try to record the peer's job as it is unlikely the peer will be in hosts. Instead, the job_command to add the peer is appended to the local job't job_data and the updated anvil-manage-striker-peers looks for that at the end of the add and sync, and records the job once the peer's UUID is in 'hosts'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-13 02:22:35 -04:00
Digimer
94d8a9c495 * Fixed a bug where finished jobs with a '0' picked-up time would be written to jobs.json.
* Updated anvil-configure-striker to use Job methods and reboot using anvil-manage-power. Also updated it to set/clear maintenance mode and mark a reboot required at the end of it's run just prior to reboot.
* Lots of log cleanup.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-07 01:29:43 -04:00
Digimer
00565b123c * Updates Tools->nice_exit to add the caller name to the exit status.
* Created Job->clear() to clear the job_picked_up_by column. Created Job->get_job_uuid() to return the job_uuid of an unfinished job matching a given job_command string (if any found).
* Updated striker->process_power to log the user out after confirming a poweroff or reboot action.
* Added anvil-daemon --startup-only to not enter the main loop and exit.
* Finished getting poweroff and reboot working (though more testing needed).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-06 01:37:08 -04:00
Digimer
423fda2ad6 * Fixed a bug in anvil-daemon where rebooting was clearing the reboot-needed flag.
* Finished (for now) adding support for monitoring jobs while a node is in maintenance mode!
* Cleaned up the display of job data and redid how buttons (real and classed links) are displayed to be consistent.
* Fixed a bug in anvil-daemon where a disconnect wasn't being called between loops, causing DB connections to pile up.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-31 02:40:49 -04:00
Digimer
60584b8cee * Created Database->get_jobs() to be a more general way to retrieve pending and recently finished jobs.
* Started adding the display of running and recently finished jobs to Striker when in maintenance mode. Still lots to do.
* Started working on the logic for what will soon be Words->decypher_string in anvil-daemon to process strings stored as '<key>,!!<name1>!<value1>!!,...,!!<nameN>!<valueN>!!'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-28 01:54:49 -04:00
Digimer
633da25d07 * Fixed a bug in Database->connect where an empty 'database::<key>' would cause an error. Updated Database->disconnect to delete the 'database' hash key as part of the same fix.
* Renamed 'database::locking' to 'sys::database::locking' to avoid collisions with 'database' keys.
* Fixed a problem with System->call where reidrects were missing the Proc::Simple method name.
* Updated anvil-daemon to check if there is no database connections on start-up, run prep-database if not, and try connecting again. If it still fails, exit. Also updated the main loop to reconnect to the database(s) and skip if non are available. Did more work on the keep_running() function.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-24 00:52:56 -04:00
Digimer
51c9d2952d * Fixed a bug with caching database handles in the recent database updates causing Database->write to stop working.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-16 19:34:37 -04:00
Digimer
fe33fbb239 * Created System->maintenance_mode to add "maintenance mode" support to Striker.
* Continuing work on adding the system update job support to Striker. Untested/Incomplete.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-16 02:38:58 -04:00
Digimer
6aa74d3d96 * Updated Database->_test_access() to use the DBD 'ping' method, and attempt a reconnect of failure.
* Updated Database->connect to take a specific UUID to attempt a connection to.
* Renamed some old 'sys::x' variables related to the database to 'sys::database::x' to conform better to coding standards.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-15 16:57:57 -04:00
Digimer
0fa3c42f2f * Fixed a bug where setting the debug level to 3 caused a deep recursion and a system hang.
* Update Anvil::Tools->new() to access the parameters 'log_level', 'log_secure' and 'debug', streamlining the frequent calls to $anvil->Log->level and ->secure in program startup, and allowing the values to take effect during the ->new constructor.
* Passed 'debug' to child method calls in more places (still more to do though).
* Fixed a bug where 'test_table' wasn't set in the right place, causing the database to try to initialize repeatedly.
* Made Database->archive_database only run if called with root access.
* Now the number of database connections are stored in 'sys::db_connections' instead of checking the returned number, and that is cleared on disconnect.
* Started working more on 'anvil-daemon', including adding support for System->call being taking 'background', 'stderr_file' and 'stdout_file' paramters which, when set, used Proc::Simple to background the process.
* Did some more work on database archiving, though still far from done.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-01 02:06:16 -04:00
Digimer
458ef94851 * Fixed a bug where the config page would be shown while a job was pending.
* Fixed a bug in Database->insert_or_update_users() wasn't handling the user's session salt.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-07-18 01:47:52 -04:00
Digimer
1539ef72a8 * Fixed how passwords are supressed in logs.
* Fixed config step 2 display.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-07-11 14:07:07 -04:00
Digimer
ee649ff2dd * Started work on the Striker sync function. It shows (without formatting yet) the ways the local dashboard can be accessed.
* Added another check and better error handling to Template->get() to print a more useful message if a template is found but fails to parse.
* Moved some strings into the words file.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-07-07 01:31:40 -04:00
Digimer
3fce021498 * Making good progress on the internal login/account methods. The user can now log in, though the first reload logs them right back out. Lots still broken.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-27 01:28:13 -04:00
Digimer
0272ba8b80 * Moved the network default values into the main defaults hash.
* Fixed a bug in Database->insert_or_update_network_interfaces where independent interfaces (not under a bridge or a bond) were not being saved.
* Continued working on improving Striker's network config jquery/form functions.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-15 02:17:20 -04:00
Digimer
76725e9160 * Fixed a bug in Database->insert_or_update_bonds() that caused unneeded UPDATE.
* Fixed anvil-update-states to record the bond UUID for interfaces already in bonds.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-07 23:15:43 -04:00
Digimer
d600a635d4 * Made 'network_interface_bridge_uuid' allow NULL entries allow NULL entries.
* Fixed a sanity check in Database->insert_or_update_network_interfaces.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-06 18:43:56 -04:00
Digimer
67c4f50bdc * Updated Database->insert_or_update_network_interfaces() to work like other similar methods. Also re-enabled NULL for 'network_interface_bond_uuid' as an empty string can't be used on a 'uuid' column type and it's not always set.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-06 18:31:19 -04:00
Digimer
870a2eb803 * Updated the SQL schema to convert all 'boolean' to 'integer' (using 0 for false). Also made all columns NOT NULL, given they should all always have an empty string. The logic behind this was to standardize the code and avoid bugs.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-04 13:12:32 -05:00
Digimer
39c06f364f * Got tools/anvil-configure-striker updating the system passwords now.
* Updated System->get_ips to only record 'sys::mac::<mac>::iface' for actual interfaces, not bonds.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-06-02 00:22:40 -05:00
Digimer
b0232c6180 * Made the sysfs directory references static.
* Fixed a typo in Database->insert_or_update_bridges().
* Added 'bond_mac_address' and 'bond_operational' to the bonds table.
* Gathered data needed for bonds in tools/anvil-update-states and got bonds recording successfully (but more testing needed).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-29 17:57:55 -04:00
Digimer
66f280c9a7 * Added Database->insert_or_update_bridges() and finished Database->insert_or_update_bonds (both need testing still).
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-25 00:10:29 -04:00
Digimer
d6c94e9d28 * Started working on Database->insert_or_update_bonds.
* Changed the database table ip_address_subnet to ip_address_subnet_mask for clarity.
* Added a check to tools/anvil-update-states to update bond entries instead of just interface entries (also not done).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-24 03:02:27 -04:00
Digimer
a6e50f15a9 * Updated System->get_ips() to determine which interface is the default gateway and set the gateway IP and active DNS servers.
* Created (but not yet tested) Database->insert_or_update_ip_addresses() and the associated ip_addresses database table to store IP address details and reference the interface they're on.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-23 03:50:30 -04:00
Digimer
6f3537807a * Added Account->read_details to collect data on the given user name.
* Updated Account->encrypt_password to return a hash reference with keys matching to database column names for consistency sake,
* Renamed 'users -> user_password' to 'user_password_hash' for clarity.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-19 00:01:24 -04:00
Digimer
04cbec30a0 * Created Database->insert_or_update_users() to add/edit users.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-17 00:45:20 -04:00
Digimer
5f06752167 * Finished updating tools/anvil-prep-database to inject the new, local database entry into anvil.conf. This tool works again.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-05-01 12:26:41 -04:00
Digimer
a89fb24adf * Changed the Storage->copy() 'target' parameter to 'target_file' to avoid confusion with the often-used 'target' parameter for connecting to remote machines.
* Changed 'database::...' so that 'x' is now the database host's UUID instead of a simple integer. This will simplify sync'ing configs. Also removed default entries, and made it so that anvil-prep-database injects the local config during first setup. Renamed Database->get_local_id to get_local_uuid and changed the 'id' parameter to 'uuid'. Changed Database->initialize's 'id' parameter to 'host_uuid'. The Database->query, Database->write, Database->_mark_database_as_behind and Database->_find_behind_databases methods had their 'id' parameter changed to 'uuid'.
* Added the 'remote_user' parameter to Get->anvil_version, System->ping and System->change_shell_user_password for conencting to remote targets.
* Added the 'remote_user' parameter to all internal Remote->call uses.
* Updated Storage->backup, Storage->copy_file, Storage->make_directory,

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-04-30 11:43:30 -04:00
Digimer
a294c6c4fa * Updated the database components to use the name 'anvil' and the user 'admin'. The 'database::user' and 'database::name' variables are still supported, but now hidden.
* Fixed a bug where some '$anvil->{}' variables should have been '$anvil->data->{}'.
* Started merging message keys on 'error_xxxx', 'warning_xxxx', etc.
* The anvil-configure-network now configures the network. Commented out, the tool can reconfigure the entire network without a reboot, but a current issue with the post-configured system refusing to use the allocated interface as the default gateway is to be reviewed at a future time. For now, a closing reboot will be issued.
* Started creating 'anvil-change-password' that will update passwords, including apache (and configure .htpasswd when needed).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-04-13 19:55:34 -04:00
Digimer
e4f7bcf661 * Created Storage->backup() that makes a backup of the given file under the Anvil! backup directory with a time-stamped suffix and preserving the original directory path.
* Got anvil-configure-network writing out the new network config properly, but renaming already-active interfaces isn't working yet.
* Updated System->get_ips() to record the interface name of a given network by MAC address using 'sys::mac::<mac_address>::iface'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-04-05 02:25:56 -04:00
Digimer
d6846841a2 * Added the 'job_status' column to the 'jobs' table where progress to be shown to users is stored.
* Updated anvil-configure-network to use Database->insert_or_update_jobs().

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-04-02 01:03:28 -04:00