The resync of the databases was originally designed (on m2) with the expextation that any given column would have only one change per 'modified_date' time. That was never a great approach, but it worked in m2 and just bit me on m3. With job processing, for an example, the job_progress will change repeatedly in one pass, all with the same 'modified_date'. So only one record per run would resync. To fix this, the plan is to drop 'history_id' (and the procedure/trigger in pgsql to copy INSERT and UPDATEs to the history schema). The new plan is to use 'change_uuid' with a per-transaction UUID created in Database so that the per-DB 'history_id' is replaced with a per-update/insert UUID in 'change_uuid'. This will become the unique record used to sync databases, instead or 'modified_date'. To keep things consistent, 'modified_date' was renamed to 'change_date' to match 'change_uuid'. This work is very much "in progress" and not finished.
This commit also changes Get->uuid to use UUID::Tiny to create v4 UUIDs instead of making making a system call to 'uuidgen'. This sped up UUID generation by almost 100x.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the Database module to not sort or reorder the 'core_tables' array, and reordered them in the hash they're declared in.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added a check to all 'Database->insert_or_update_*' methods to check if the passed-in reference UUID was found and return an empty string if not.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database->insert_or_update_jobs() to also use the job_command when looking for an existing job (when a specific job_uuid was not included).
* Fixed a bug with a missing ? in striker->add_sync_peer function. Also updated it to not try to record the peer's job as it is unlikely the peer will be in hosts. Instead, the job_command to add the peer is appended to the local job't job_data and the updated anvil-manage-striker-peers looks for that at the end of the add and sync, and records the job once the peer's UUID is in 'hosts'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Bumped the RPM spec file to 15, though haven't actually rolled the new RPM yet. Also added 'htop' as an anvil-core dependencies.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil-configure-striker to use Job methods and reboot using anvil-manage-power. Also updated it to set/clear maintenance mode and mark a reboot required at the end of it's run just prior to reboot.
* Lots of log cleanup.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Job->clear() to clear the job_picked_up_by column. Created Job->get_job_uuid() to return the job_uuid of an unfinished job matching a given job_command string (if any found).
* Updated striker->process_power to log the user out after confirming a poweroff or reboot action.
* Added anvil-daemon --startup-only to not enter the main loop and exit.
* Finished getting poweroff and reboot working (though more testing needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got the webui portion of requesting a poweroff and reboot done, but still working on finishing anvil-manage-power (work on which lead to the above improvement).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created the Anvil::Tools::Jobs module to handle general job processing task. Moved 'update_progress' from tools/anvil-update-system to it and generalized it.
* Added some missing CDATA wrappers to the words XML file strings with '>' in it.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added '--refresh-json' to anvil-daemon that auto-selects '--run-once', '--main-loop-only' and '--no-start'.
* Updated anvil-update-system to not go more than a second between updates to the progress (save for when we're holding on data from 'dnf').
Signed-off-by: Digimer <digimer@alteeve.ca>
* Made it so that anvil-daemon won't restart when on-disk version has changed while jobs are still running.
* Made it so that anvil-update-system reloads systemctl after the update finishes to pickup changes in updated system daemons.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Finished (for now) adding support for monitoring jobs while a node is in maintenance mode!
* Cleaned up the display of job data and redid how buttons (real and classed links) are displayed to be consistent.
* Fixed a bug in anvil-daemon where a disconnect wasn't being called between loops, causing DB connections to pile up.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil-update-system. to set and then clear maintenance mode.
* Added the files needed for jquery-ui v1.12.1
* Added unused and xcf format icons to the alteeve skin.
* Disabled all 'postun' scriptlets in anvil.spec.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started adding the display of running and recently finished jobs to Striker when in maintenance mode. Still lots to do.
* Started working on the logic for what will soon be Words->decypher_string in anvil-daemon to process strings stored as '<key>,!!<name1>!<value1>!!,...,!!<nameN>!<valueN>!!'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed 'database::locking' to 'sys::database::locking' to avoid collisions with 'database' keys.
* Fixed a problem with System->call where reidrects were missing the Proc::Simple method name.
* Updated anvil-daemon to check if there is no database connections on start-up, run prep-database if not, and try connecting again. If it still fails, exit. Also updated the main loop to reconnect to the database(s) and skip if non are available. Did more work on the keep_running() function.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed tools/anvil-clear-reboot to tools/anvil-reboot-needed and changed it to behave like anvil-maintenance-mode.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created tools/anvil-clear-reboot to clear the "reboot needed" flag. Also created, but not yet using (and may not use) units/anvil-boot-time.service.
* Started work on having jobs show their data via JSON / jquery.
* Updated anvil-update-system to record messages indicating the progress so far.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database->connect to take a specific UUID to attempt a connection to.
* Renamed some old 'sys::x' variables related to the database to 'sys::database::x' to conform better to coding standards.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the RPM to .13 to disable postun's disabling of postgres, which breaks Anvil! software using the database during RPM updates.
* Fixed a logging bug where the number of DB connections was not inserting the number properly.
* Fixed exits in tools/anvil-prep-database.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started adding support for updating the striker (and later, all) systems. This will be handled by the in-progress tools/anvil-update-system program.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-manage-striker-peers working properly (so far).
* Updated anvil-prep-database to call anvil-manage-striker-peers, but testing still needed.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Update Anvil::Tools->new() to access the parameters 'log_level', 'log_secure' and 'debug', streamlining the frequent calls to $anvil->Log->level and ->secure in program startup, and allowing the values to take effect during the ->new constructor.
* Passed 'debug' to child method calls in more places (still more to do though).
* Fixed a bug where 'test_table' wasn't set in the right place, causing the database to try to initialize repeatedly.
* Made Database->archive_database only run if called with root access.
* Now the number of database connections are stored in 'sys::db_connections' instead of checking the returned number, and that is cleared on disconnect.
* Started working more on 'anvil-daemon', including adding support for System->call being taking 'background', 'stderr_file' and 'stdout_file' paramters which, when set, used Proc::Simple to background the process.
* Did some more work on database archiving, though still far from done.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Get->users_home() to default to return the hore directory for the user running the program.
* Updated Remote->call() to start working on handling timeouts.
* Updated Storage->change_owner(), ->make_directory() and ->write_file() to default the the user and group running the program.
* Fixed a bug in home reporting the MAC address of NICs when confirming configuration of Striker. Also changed showing the domain to the hostname.
* Got more work done on sync peers.
* Updated the RPM spec file to install on Fedora 28.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Set Log->entry to chmod the log file to 666 when the file is opened to ensure apache can write to it.
* Fixed a string replacement variable name.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated tools/anvil-prep-database to use a daemon name from Tools.pm to more easily support later postgres version upgrades.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Cleaned up 'button' CSS class links and submit CSS to match.
* Re-enabled reboot at the end of a tools/anvil-configure-striker run.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added error messages to Striker configuration forms.
* Fixed a bug in home->get_network_details() function to handle single IPs in network.xml.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Database->insert_or_update_network_interfaces where independent interfaces (not under a bridge or a bond) were not being saved.
* Continued working on improving Striker's network config jquery/form functions.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a typo in Database->insert_or_update_bridges().
* Added 'bond_mac_address' and 'bond_operational' to the bonds table.
* Gathered data needed for bonds in tools/anvil-update-states and got bonds recording successfully (but more testing needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Changed the database table ip_address_subnet to ip_address_subnet_mask for clarity.
* Added a check to tools/anvil-update-states to update bond entries instead of just interface entries (also not done).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created (but not yet tested) Database->insert_or_update_ip_addresses() and the associated ip_addresses database table to store IP address details and reference the interface they're on.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed tools/anvil-configure-network to tools/anvil-configure-striker given that it will also now update system passwords.
* Started working on tools/anvil-update-states to properly handle a Striker with already-configured networking.
* Cleaned up tools/anvil-change-password.
* Fixed a bug in Storage->update_config to set the ownership of anvil.conf to 'apache:apache' so that the web server can read it.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Account->encrypt_password to return a hash reference with keys matching to database column names for consistency sake,
* Renamed 'users -> user_password' to 'user_password_hash' for clarity.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added user_algorithm and user_hash_count to the new users database table so that we can remember how a hash was generated, should it be changed down the road.
* Made the salt length configurable by the user (as well as the algorithm and loop count).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created the new Account.pm module for handling account related tasks.
* Moved 'sys::stty' to 'sys::terminal:stty'.
* Created Get->_salt which generates a (pseudo) random salt for internal passwords and Get->_wrap_to that checks the current terminal width.
* Added the parameter 'redirect_stderr' (default '1') to System->call to optionally not redirect STDERR to STDOUT (needed for tput to be useful as STDERR is a terminal of sorts).
* Disabled System->change_apache_password (to be removed entirely later).
* Created Words->_wrap_string to wrap text strings to the active terminal width.
* Added the user's table to the core SQL. Also deleted the empty Anvil/Tools.sql file.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Changed 'database:❌:...' so that 'x' is now the database host's UUID instead of a simple integer. This will simplify sync'ing configs. Also removed default entries, and made it so that anvil-prep-database injects the local config during first setup. Renamed Database->get_local_id to get_local_uuid and changed the 'id' parameter to 'uuid'. Changed Database->initialize's 'id' parameter to 'host_uuid'. The Database->query, Database->write, Database->_mark_database_as_behind and Database->_find_behind_databases methods had their 'id' parameter changed to 'uuid'.
* Added the 'remote_user' parameter to Get->anvil_version, System->ping and System->change_shell_user_password for conencting to remote targets.
* Added the 'remote_user' parameter to all internal Remote->call uses.
* Updated Storage->backup, Storage->copy_file, Storage->make_directory,
Signed-off-by: Digimer <digimer@alteeve.ca>
* Made logging between journald and a traditional file configurable via 'sys::log_file'. Also made the file handle unbuffered when logging to a file.
* Fixed a bug with loading the anvil.conf config file in a few locations.
* Created System->stty_echo() to handle enabling/disabling shell echo, and added restoring the echo to Tools->catch_sig.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug with handling ssh fingerprints (and removed comments going to the known_hosts file).
* Added more nested debug parameter passing when methods call other methods (though more work is needed to catch up)
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Storage->read_file and Storage->write_file to support reading and writing on remote systems (untested though)
* Created System->change_shell_user_password() that changes a shell user's password by manually generating an sha512 salted hash of the given password and uses the resulting hash to modify the target user's password, so the password should never be visible in the process list. Works on both local and remote systems, though it still needs testing.
* Created Storage->rsync() to handle moving files between the local and a remote system.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where some '$anvil->{}' variables should have been '$anvil->data->{}'.
* Started merging message keys on 'error_xxxx', 'warning_xxxx', etc.
* The anvil-configure-network now configures the network. Commented out, the tool can reconfigure the entire network without a reboot, but a current issue with the post-configured system refusing to use the allocated interface as the default gateway is to be reviewed at a future time. For now, a closing reboot will be issued.
* Started creating 'anvil-change-password' that will update passwords, including apache (and configure .htpasswd when needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-configure-network writing out the new network config properly, but renaming already-active interfaces isn't working yet.
* Updated System->get_ips() to record the interface name of a given network by MAC address using 'sys::mac::<mac_address>::iface'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-configure-network setting the new hostname.
* Updated anvil-configure-network to exit only if the job was picked up by a still-running PID.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got tools/anvil-configure-network to exit if another instance is running. If not, it now starts (barely) the process of configuring the network.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed some logging in Get->cgi() and generally cleaned up logging levels.
* Got striker to the point where the job to reconfigure the network is saved in the database and the dashboard goes offline until it is done.
* Created the start of the new anvil-configure-network tool.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added 'file' and 'line' arguments to the Database->insert_or_update_X methods to allow for the original caller's file and line number to be recorded in the SQL call logs.
* Cleaned up how logging to 'anvil.log' logging is handled.
* Updated anvil-update-states to ignore libvirt bridges and to manually set the speed and duplex of virtio network based interfaces.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug with resync, but others remain as resync is incomplete (at least for network_interfaces).
* Currently, tools/anvil-update-states is broken while working on the above issue.
* Reworked the jobs table and removed the units/anvil-jobs.service unit. Jobs will be invoked and backgrounded in all calls.
* Started adding missing hidden form fields.
* Updated the 'server' OCF resource agent version and metadata.
Signed-off-by: Digimer <digimer@alteeve.ca>
