* Fixed some bugs in Database->insert_or_update_files() and the 'files' table/procedure in the SQL schema.
* Got more work done on anvil-manage-files.
* Created Job->get_job_details().
* Added an executable check for files in Storage->scan_directory().
* Cleaned up some logging and switched to Job->get_job_details() in anvil-update-states and striker-configure-host.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Get->uuid to take the new 'short' parameter that, when passed, asks for just the first 8 bytes of the UUID string.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started work of "Files" (replacement for the media library), including database tables, planned sync flow and web UI.
* Added a check for the /mnt/shared directories and create them as needed in the periodic anvil-daemon checks.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed a couple Striker-only tools to use the 'striker' prefix instead of 'anvil'.
* Updated the core_tables list.
* Renamed 'sys::log::main' to 'sys::log::file'.
* Fixed some "Back" and "Refresh" links.
* Started planning out the file sync system.
* Started work on the Anvil! setup / host prep system.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Job->html_list() to return the list of running or recently completed jobs, and used it instead of the code formarly used in 'striker' when Striker was unavailable.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil.sql to add the new tables needed for alert mail delivery.
* Update anvil.sql and Database->initialize to now default the user to 'admin' and swap that out if needed, instead of using the #!variable!user!#' replacement variable.
* Started updating anvil.spec for EL8.
* Added support for 'striker::repo::extra-packages' which users can use to add additional packages to the Striker repositories.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed Alert->register_alert() to ->register() and updated it to take 'clear_alert' and used it and the alert level to set the title automatically if not set by the user.
* Updated Log->_adjust_log_level() to record when the user set the log level at the command line so that invoked child processes get called with the same log level switch.
* Got the framework for actually calling scan agents in scancore in place. Untested so far.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil-daemon to re-read the main words file on each loop.
* Updated scancore to read and purge each scan agent's words file between invocations.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got scancore scanning the agents directory, and properly holding on startup until at least one database is available (instead of exiting), and holding on startup until the local system is configured.
* Created the skeleton of the first scan agent; scan-network.
* Fixed a bug in Storage->check_md5sums() where dynamically loaded modules, loaded after the initial md5sum calcs, would cause the calling daemon to exit (possibly on every invocation).
* Created the scancore.README that will eventually be the main scan agent guide / API document.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started expanding Alert->register_alert() to actually implement it.
* Improved handling errors in Words->key().
* Started work on Striker's "Anvil!" menu section. Also cleaned up the power handling.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Log->entry() to accept 'print => [0|1]' to send a log message to STDOUT (minus prefix) to avoid tools that were repeatedly calling print and Log->entry back to back.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Words->parse_banged_string where some variable strings were not being cleared, causing infinite loops.
* Added job progress reporting in striker-manage-install-target, and made it only refresh the RPM repo when '--refresh' is specified (with --force now forcing the issue). This was done to allow adding it into anvil-daemon in such a way that it would only update the RPM repo once a day.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Get->anvil_version where the version of local systems and remote systems differed in closing new lines.
* Fixed a bug in Database->insert_or_update_variables() where the 'debug' parameter wasn't working.
* Renamed System->determine_host_type -> System->get_host_type.
* Fixed a bug in System->get_uptime where there was a newline after the uptime integer.
* Updated anvil-daemon to track and record the state of the Install Target feature on Striker dashboards.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Work on anvil-manage-install-target to enable/disable dhcpd. Also to only refresh the RPM repo periodically. Fixed a bug where it always reported that the kickstart files were not updated, even when they were.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-manage-install-target finished creating config files and enabling daemons needed for PXE. Still untested in function though.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Storage->read_file() where the last newline wasn't always being faithfully recorded.
* Created System->restart_daemon (as opposed to ->reload_daemon).
* Got creating/updating dhcpd.conf / dhcpd working in tools/anvil-manage-install-target.
Signed-off-by: Digimer <digimer@alteeve.ca>
* After much time wasted chasing a dnf bug (https://bugzilla.redhat.com/show_bug.cgi?id=1641947), tools/anvil-manage-install-target now populates the <DOCROOT>/<os_type>/<os_arch>/os/Packages/ directory with needed RPMs.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added 'sys::database::failed_connection_log_level' to allow silencing of log messages when a Striker peer database is not available.
* Started updating the .spec for the new release to add supported packages needed for PXE/dhcp/tftpboot.
* Added to repo tftpboot files as pulling them out of the packages and moving them into the right place relative to the modest size of adding them directly to our source wasn't justified.
* Created the still very very early 'tools/anvil-manage-firewall' tool.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Jobs->update_progress to take 'picked_up_by' as an optional parameter, defaulting to '$$' (the caller's PID).
* Created System->get_uptime() to return the current uptime in seconds.
* Added a delay to anvil-manage-power to not proceed with a reboot if the uptime is less than 600 seconds. This way, if any future bug causes an infinite reboot, there will be more time to determine what's wrong and debug the system between reboots.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Cleaned up some logging.
* Made the "Reload" buttons work more sensibly and cleaned up some webui display stuff.
* Got deleting peers mostly working (well, it works, but then it goes into a loop thinking it needs to resync the now-gone database until the daemon restarts).
* Fixed a race condition bug where if a job exited between the time that anvil-daemon got a list of PIDs and when it checked to see if that specific pid was alive, a job that actually completed could be restarted.
* Added a loop check to anvil-manage-striker-peers where it would hold until a database connection to the newly added peer was available, preventing a condition where re-adding a peer (and so the host_uuid is in hosts) cause the job belonging to the peer to be recorded locally and then never synced to the peer.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added 'sys::log_date' which controls if the date and time is pre-pended to log entries.
* Created Get->host_name() which takes a host UUID and returns the 'host_name' from the 'hosts' table, if found.
* Cleaned up some HTML templates and logging.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database methods that can take the caller's file or line to append the local file and line.
* Updated Jobs methods to call Database->insert_or_update_jobs().
* Updated the remaining UPDATEs of 'jobs' table to use 'Database->refresh_timestamp()' for the 'modified_date'.
Signed-off-by: Digimer <digimer@alteeve.ca>
The resync of the databases was originally designed (on m2) with the expextation that any given column would have only one change per 'modified_date' time. That was never a great approach, but it worked in m2 and just bit me on m3. With job processing, for an example, the job_progress will change repeatedly in one pass, all with the same 'modified_date'. So only one record per run would resync. To fix this, the plan is to drop 'history_id' (and the procedure/trigger in pgsql to copy INSERT and UPDATEs to the history schema). The new plan is to use 'change_uuid' with a per-transaction UUID created in Database so that the per-DB 'history_id' is replaced with a per-update/insert UUID in 'change_uuid'. This will become the unique record used to sync databases, instead or 'modified_date'. To keep things consistent, 'modified_date' was renamed to 'change_date' to match 'change_uuid'. This work is very much "in progress" and not finished.
This commit also changes Get->uuid to use UUID::Tiny to create v4 UUIDs instead of making making a system call to 'uuidgen'. This sped up UUID generation by almost 100x.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the Database module to not sort or reorder the 'core_tables' array, and reordered them in the hash they're declared in.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added a check to all 'Database->insert_or_update_*' methods to check if the passed-in reference UUID was found and return an empty string if not.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database->insert_or_update_jobs() to also use the job_command when looking for an existing job (when a specific job_uuid was not included).
* Fixed a bug with a missing ? in striker->add_sync_peer function. Also updated it to not try to record the peer's job as it is unlikely the peer will be in hosts. Instead, the job_command to add the peer is appended to the local job't job_data and the updated anvil-manage-striker-peers looks for that at the end of the add and sync, and records the job once the peer's UUID is in 'hosts'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Bumped the RPM spec file to 15, though haven't actually rolled the new RPM yet. Also added 'htop' as an anvil-core dependencies.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil-configure-striker to use Job methods and reboot using anvil-manage-power. Also updated it to set/clear maintenance mode and mark a reboot required at the end of it's run just prior to reboot.
* Lots of log cleanup.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Job->clear() to clear the job_picked_up_by column. Created Job->get_job_uuid() to return the job_uuid of an unfinished job matching a given job_command string (if any found).
* Updated striker->process_power to log the user out after confirming a poweroff or reboot action.
* Added anvil-daemon --startup-only to not enter the main loop and exit.
* Finished getting poweroff and reboot working (though more testing needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got the webui portion of requesting a poweroff and reboot done, but still working on finishing anvil-manage-power (work on which lead to the above improvement).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created the Anvil::Tools::Jobs module to handle general job processing task. Moved 'update_progress' from tools/anvil-update-system to it and generalized it.
* Added some missing CDATA wrappers to the words XML file strings with '>' in it.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added '--refresh-json' to anvil-daemon that auto-selects '--run-once', '--main-loop-only' and '--no-start'.
* Updated anvil-update-system to not go more than a second between updates to the progress (save for when we're holding on data from 'dnf').
Signed-off-by: Digimer <digimer@alteeve.ca>
* Finished (for now) adding support for monitoring jobs while a node is in maintenance mode!
* Cleaned up the display of job data and redid how buttons (real and classed links) are displayed to be consistent.
* Fixed a bug in anvil-daemon where a disconnect wasn't being called between loops, causing DB connections to pile up.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started adding the display of running and recently finished jobs to Striker when in maintenance mode. Still lots to do.
* Started working on the logic for what will soon be Words->decypher_string in anvil-daemon to process strings stored as '<key>,!!<name1>!<value1>!!,...,!!<nameN>!<valueN>!!'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed 'database::locking' to 'sys::database::locking' to avoid collisions with 'database' keys.
* Fixed a problem with System->call where reidrects were missing the Proc::Simple method name.
* Updated anvil-daemon to check if there is no database connections on start-up, run prep-database if not, and try connecting again. If it still fails, exit. Also updated the main loop to reconnect to the database(s) and skip if non are available. Did more work on the keep_running() function.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added a reboot button to managing the Striker. This button and the text associated with it reflects the need for the reboot.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed tools/anvil-clear-reboot to tools/anvil-reboot-needed and changed it to behave like anvil-maintenance-mode.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created tools/anvil-clear-reboot to clear the "reboot needed" flag. Also created, but not yet using (and may not use) units/anvil-boot-time.service.
* Started work on having jobs show their data via JSON / jquery.
* Updated anvil-update-system to record messages indicating the progress so far.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database->connect to take a specific UUID to attempt a connection to.
* Renamed some old 'sys::x' variables related to the database to 'sys::database::x' to conform better to coding standards.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the RPM to .13 to disable postun's disabling of postgres, which breaks Anvil! software using the database during RPM updates.
* Fixed a logging bug where the number of DB connections was not inserting the number properly.
* Fixed exits in tools/anvil-prep-database.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started adding support for updating the striker (and later, all) systems. This will be handled by the in-progress tools/anvil-update-system program.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-manage-striker-peers working properly (so far).
* Updated anvil-prep-database to call anvil-manage-striker-peers, but testing still needed.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Update Anvil::Tools->new() to access the parameters 'log_level', 'log_secure' and 'debug', streamlining the frequent calls to $anvil->Log->level and ->secure in program startup, and allowing the values to take effect during the ->new constructor.
* Passed 'debug' to child method calls in more places (still more to do though).
* Fixed a bug where 'test_table' wasn't set in the right place, causing the database to try to initialize repeatedly.
* Made Database->archive_database only run if called with root access.
* Now the number of database connections are stored in 'sys::db_connections' instead of checking the returned number, and that is cleared on disconnect.
* Started working more on 'anvil-daemon', including adding support for System->call being taking 'background', 'stderr_file' and 'stdout_file' paramters which, when set, used Proc::Simple to background the process.
* Did some more work on database archiving, though still far from done.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where "form::error_massage" was default to ' ' which caused simple checks to thing there was an error when all was fine.
* Got the "add new sync peer" form confirmation box displaying and cleaned up the CSS a bit.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Get->users_home() to default to return the hore directory for the user running the program.
* Updated Remote->call() to start working on handling timeouts.
* Updated Storage->change_owner(), ->make_directory() and ->write_file() to default the the user and group running the program.
* Fixed a bug in home reporting the MAC address of NICs when confirming configuration of Striker. Also changed showing the domain to the hostname.
* Got more work done on sync peers.
* Updated the RPM spec file to install on Fedora 28.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Set Log->entry to chmod the log file to 666 when the file is opened to ensure apache can write to it.
* Fixed a string replacement variable name.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added another check and better error handling to Template->get() to print a more useful message if a template is found but fails to parse.
* Moved some strings into the words file.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Moved templates around different files to clean things up.
* Moved the "back" and "refresh" icons over to the right by the logo, and added a new icon for handling mail, alerts, install targets and manifests.
* Started work on the Striker configuration sub-page.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where the old style '#!replace!...!#' replacement variables were not being escaped when processing variable insertions into strings.
* Made the body variable be stored in 'form::body' instead of passing around the '$body' variable.
* Created a set of new icons.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added error messages to Striker configuration forms.
* Fixed a bug in home->get_network_details() function to handle single IPs in network.xml.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed tools/anvil-configure-network to tools/anvil-configure-striker given that it will also now update system passwords.
* Started working on tools/anvil-update-states to properly handle a Striker with already-configured networking.
* Cleaned up tools/anvil-change-password.
* Fixed a bug in Storage->update_config to set the ownership of anvil.conf to 'apache:apache' so that the web server can read it.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added user_algorithm and user_hash_count to the new users database table so that we can remember how a hash was generated, should it be changed down the road.
* Made the salt length configurable by the user (as well as the algorithm and loop count).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Changed 'database:❌:...' so that 'x' is now the database host's UUID instead of a simple integer. This will simplify sync'ing configs. Also removed default entries, and made it so that anvil-prep-database injects the local config during first setup. Renamed Database->get_local_id to get_local_uuid and changed the 'id' parameter to 'uuid'. Changed Database->initialize's 'id' parameter to 'host_uuid'. The Database->query, Database->write, Database->_mark_database_as_behind and Database->_find_behind_databases methods had their 'id' parameter changed to 'uuid'.
* Added the 'remote_user' parameter to Get->anvil_version, System->ping and System->change_shell_user_password for conencting to remote targets.
* Added the 'remote_user' parameter to all internal Remote->call uses.
* Updated Storage->backup, Storage->copy_file, Storage->make_directory,
Signed-off-by: Digimer <digimer@alteeve.ca>
* Made logging between journald and a traditional file configurable via 'sys::log_file'. Also made the file handle unbuffered when logging to a file.
* Fixed a bug with loading the anvil.conf config file in a few locations.
* Created System->stty_echo() to handle enabling/disabling shell echo, and added restoring the echo to Tools->catch_sig.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug with handling ssh fingerprints (and removed comments going to the known_hosts file).
* Added more nested debug parameter passing when methods call other methods (though more work is needed to catch up)
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Storage->read_file and Storage->write_file to support reading and writing on remote systems (untested though)
* Created System->change_shell_user_password() that changes a shell user's password by manually generating an sha512 salted hash of the given password and uses the resulting hash to modify the target user's password, so the password should never be visible in the process list. Works on both local and remote systems, though it still needs testing.
* Created Storage->rsync() to handle moving files between the local and a remote system.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where some '$anvil->{}' variables should have been '$anvil->data->{}'.
* Started merging message keys on 'error_xxxx', 'warning_xxxx', etc.
* The anvil-configure-network now configures the network. Commented out, the tool can reconfigure the entire network without a reboot, but a current issue with the post-configured system refusing to use the allocated interface as the default gateway is to be reviewed at a future time. For now, a closing reboot will be issued.
* Started creating 'anvil-change-password' that will update passwords, including apache (and configure .htpasswd when needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-configure-network writing out the new network config properly, but renaming already-active interfaces isn't working yet.
* Updated System->get_ips() to record the interface name of a given network by MAC address using 'sys::mac::<mac_address>::iface'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-configure-network setting the new hostname.
* Updated anvil-configure-network to exit only if the job was picked up by a still-running PID.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed some logging in Get->cgi() and generally cleaned up logging levels.
* Got striker to the point where the job to reconfigure the network is saved in the database and the dashboard goes offline until it is done.
* Created the start of the new anvil-configure-network tool.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added 'file' and 'line' arguments to the Database->insert_or_update_X methods to allow for the original caller's file and line number to be recorded in the SQL call logs.
* Cleaned up how logging to 'anvil.log' logging is handled.
* Updated anvil-update-states to ignore libvirt bridges and to manually set the speed and duplex of virtio network based interfaces.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug with resync, but others remain as resync is incomplete (at least for network_interfaces).
* Currently, tools/anvil-update-states is broken while working on the above issue.
* Reworked the jobs table and removed the units/anvil-jobs.service unit. Jobs will be invoked and backgrounded in all calls.
* Started adding missing hidden form fields.
* Updated the 'server' OCF resource agent version and metadata.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added Get->anvil_version() to check the local or remote Anvil! version.
* Added a check in Database->connect() to see if a database server's Anvil! version matches the local version. If the versions don't match, the database is not used.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added the 'debug' parameter to System->call(). Also added a check to make sure that the called executable exists.
* Added the 'debug' parameter to System->start_daemon().
Signed-off-by: Digimer <digimer@alteeve.ca>
* Moved all executables to /usr/sbin/
* Made /root/anvil-backups/ the backup directory.
* Started debuging anvil-prep-database
Signed-off-by: Digimer <digimer@alteeve.ca>
