* Fixed a bug in Convert->time() where the suffix was long when it should have been short, and vice-versa.
* Updated Network->download() to check if the target file exists and, if so, to abort unless 'overwrite' is given or the existing file is 0-bytes long. Also updated it to not exit on immediate error after the wget call and instead check to see if a zero-byte file exists and remove it, if so.
* Created Validate->is_hex() to check hexadecimal strings.
* Updated Words->clean_spaces() to remove MS-DOS-style ^M cr/lf characters.
* Updated anvil-daemon to have a section for periodic tasks that run daily, and added striker-parse-oui as well as moved striker-manage-install-target refresh to that check. Also made those tools run on dashboards only.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Database->insert_or_update_bridge_interfaces() to handle updating the new bridge_interfaces database table that records which network interfaces are connected to which bridges.
* Added 'bridge_mac' and 'bridge_mtu' to the 'bridges' table.
* Started Server->map_network which will, eventually, try to map MAC addresses to IPs and record server -> vnetX -> bridge data. Made getting a server status target-dependent.
* Worked on anvil-update-states to parse and record bridge data.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Moved System->is_local to Network->is_local, and System->ping to Network->ping.
* Added a check to tools/striker-get-peer-data that will report if the target has Internet access or not.
* Cleaned up the form that prompts the user to enter their Red Hat credentials.
* Updated tools/anvil-manage-keys (and related code) to no longer distinguish by user. If a target is flagged as changed, it is removed from the root and all user's known_hosts files.
* Updated Storage->write_file() and ->update_file() to accept the 'backup' parameter to control if an file that exists is backed up before being updated/replaced.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated System->host_name to work locally and on remote targets.
* Renamed all 'hostname' instances to 'host_name' to standardize on a spelling throughout the program.
* Removed use of and dependency on 'hostname'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added a check to Remote->call() where, when a connect attempt fails because of a changed/bad key, it is reported as such to the user/logs and an entry is recorded in the state file.
* Started adding a Striker menu function showing users a list of bad keys in known_hosts files and the ability to remove old keys.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Remote->call() to detect when a connection fails because the target's known_hosts entry has changed. Still need to add the function to report this to the user.
* Fixed a bug where new-lines in Words->parse_banged_string() where a double-banged word string's variable value would cause an infinite loop.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Tools->refresh to reload anvil.conf in one call.
* Created Anvil::Tools::Network to hold network-related tasks.
** Created Network->is_remote() that tests to see if a string (containing a target) refers to the remote machine (versus a local machine). Updated all previous checks to use this new method.
** Moved Get->network_details() and Get->network() to the new Network module. Renamed Get->network() to Network->get_network().
** Made Network->get_ips() work locally and remotely.
** Created Network->find_matches() that compares two scanned machines IPs (via two previous calls to Network->get_ips())
* Created Database->manage_anvil_conf() that will add, update or remove a given database connection in a local or remote anvil.conf file.
* Fixed bugs in Storage->backup() where the bash calls were quite broken. I'm not sure how it ever worked before... x_x
* Updated anvil-daemon to not initialize a database unless it's running on dashboard. Also added a check at the startup of anvil-daemon where it will go into a loop waiting for a database to become available, re-reading anvil.conf each loop.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Remote->call where a shell call that ended in a newline would work, but throw an error and not get the return code.
* Created Database->get_job_details() which takes a job_uuid and returns the job details, if found.
* Fixed a bug in Jobs->update_progress() where 'clear' wasn't removing the old job_progress data.
* Added the parameters 'no_files' to skip stat'ing/recording non-directories, and 'search_for' which will set the parent directory in 'scan::searched' and stop scanning if found. This allows this method to act as a directory tree scanner and as a search engine.
* Created Striker->get_local_repo() that builds a repo file body suitable for adding to peers, nodes and DR hosts.
* Fixed bugs in the Striker WebUI related to initializing a target node / DR host.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated striker to no longer try to SSH to a remote machine. To enable this, we'd have to give apache a shell and an SSH key, which is dumb and dangerous when considered.
* Created tools/striker-get-peer-data which is meant to be invoked as the 'admin' user (via a setuid c-wrapper). It collects basic data about a target machine and reports what it finds on STDOUT. It gets the password for the target via the database.
* Updated anvil-daemon to check/create/update setuid c-wrapper(s), which for now is limited to call_striker-initialize-host.
* Created Anvil/Tools/Striker.pm to store Striker web-specific methods, including get_peer_data() which calls tools/striker-initialize-host via the setuid admin call_striker-initialize-host c-wrapper.
* In order to allow striker via apache to read a peer's anvil.version, which it can no longer do over SSH, any connection to a peer where the anvil.version is read is cached as /etc/anvil/anvil.<peer>.version. When Get->anvil_version is called as 'apache', this file is read instead.
* Updated Database->resync_databases() and ->_find_behind_databases() to ignore the 'states' table.
* Created tools/striker-initialize-host which will be called as a job to initialize a node/dr host.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created tools/anvil-check-memory to report how much RAM is used by a given program.
* Added documentation for some previously undocumented methods.
* Updated Database->archive_database() to take the 'tables' parameter.
* Updated Storage->scan_directory() to record a directory's mode and type, even when recursive isn't used.
* Finished System->check_memory().
* Updated ocf:alteeve:server to now NOT stop a DRBD resource unless 'stop_drbd_resources'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created System->active_lv() that, surprise, activates an inactive logical volume. Also created ->check_storage() that parses out the LVM data.
* Fixed a bug in tools/fence_pacemaker that was preventing it from compiling and running.
* Updated ocf:alteeve:server to validate the target server's storage.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started a re-write of the ocf:alteeve:server resource agent. It has bugs and is missing logic, and trying to clean it up given most of the bugs come from trying to clean up the original agent doesn't make sense. Moving much of the logic into module methods as the functions will be needed elsewhere anyway.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started working on ocf:alteeve:server to make it smarter (and more patient) when bringing a DRBD resource up so we don't get false failures when we hit a race.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where Get->host_uuid() wasn't reading from the host.uuid file.
* Updated Remote->call() to record a target's fingerprint when needed.
* The ocf:alteeve:server resource agent now properly stopps a server and the corresponding DRBD resource.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database->insert_or_update_jobs() to use job_data when looking for a job uuid. Also fixed logging and adapted 'jobs::X' variable feeding to prevent them being undefined. Also made the search find jobs with the program name anywhere in the string, instead of just the start of the strong.
* Started work on Jobs->update_progress() to handle updating downloading file information.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added to Convert->time the 'translate' parameter that controls if the returned string is already translated or not.
* Updated Storage->change_mode and ->change_owner to rename the parameter 'target' to 'path' to help prevent future confusion woth most other instances of the 'target' parameter meaning a target machine.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started working on Convert->time().
* Changed anvil-manage-files skip /mnt/shared/temp when looking for files to add to the database.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the files table to add the file_mtime. In the future, if two versions of the same file exists on different machines, the one with the more recent mtime will be copied over the others.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Finished getting anvil-manage-files to find and process new files in /mnt/shared/incoming. Created a 'convert_mimetype' function to translate returned mimetype to a file type we care about.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed some bugs in Database->insert_or_update_files() and the 'files' table/procedure in the SQL schema.
* Got more work done on anvil-manage-files.
* Created Job->get_job_details().
* Added an executable check for files in Storage->scan_directory().
* Cleaned up some logging and switched to Job->get_job_details() in anvil-update-states and striker-configure-host.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Get->uuid to take the new 'short' parameter that, when passed, asks for just the first 8 bytes of the UUID string.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started work of "Files" (replacement for the media library), including database tables, planned sync flow and web UI.
* Added a check for the /mnt/shared directories and create them as needed in the periodic anvil-daemon checks.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed a couple Striker-only tools to use the 'striker' prefix instead of 'anvil'.
* Updated the core_tables list.
* Renamed 'sys::log::main' to 'sys::log::file'.
* Fixed some "Back" and "Refresh" links.
* Started planning out the file sync system.
* Started work on the Anvil! setup / host prep system.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Disabled deletion of unneeded zones.
* Updated the default BCN/SN IPs generated in striker to follow the new schema.
Firewall work continues.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added a 'test' parameter to Log->entry, Storage->make_directory and Words->key to help debug in places that Log->x may not be usable.
* Converted many $anvil->Log->x calls to print if $test to help prevent recursive loops, but not all fixed yet.
* Added the new 'host_keys' database table to the schema for a possible new feature of removing passwords in favour if machines adding peers' public keys to their authorized_hosts file.
* Cleaned up the opening calls to $anvil->Tools->new() in most tools.
* Cleaned up some variables in tools/anvil-update-states after reading their values from files (clean trailing newlines).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil.sql to add the new tables needed for alert mail delivery.
* Update anvil.sql and Database->initialize to now default the user to 'admin' and swap that out if needed, instead of using the #!variable!user!#' replacement variable.
* Started updating anvil.spec for EL8.
* Added support for 'striker::repo::extra-packages' which users can use to add additional packages to the Striker repositories.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Renamed Alert->register_alert() to ->register() and updated it to take 'clear_alert' and used it and the alert level to set the title automatically if not set by the user.
* Updated Log->_adjust_log_level() to record when the user set the log level at the command line so that invoked child processes get called with the same log level switch.
* Got the framework for actually calling scan agents in scancore in place. Untested so far.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil-daemon to re-read the main words file on each loop.
* Updated scancore to read and purge each scan agent's words file between invocations.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got scancore scanning the agents directory, and properly holding on startup until at least one database is available (instead of exiting), and holding on startup until the local system is configured.
* Created the skeleton of the first scan agent; scan-network.
* Fixed a bug in Storage->check_md5sums() where dynamically loaded modules, loaded after the initial md5sum calcs, would cause the calling daemon to exit (possibly on every invocation).
* Created the scancore.README that will eventually be the main scan agent guide / API document.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Started expanding Alert->register_alert() to actually implement it.
* Improved handling errors in Words->key().
* Started work on Striker's "Anvil!" menu section. Also cleaned up the power handling.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Log->entry() to accept 'print => [0|1]' to send a log message to STDOUT (minus prefix) to avoid tools that were repeatedly calling print and Log->entry back to back.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Words->parse_banged_string where some variable strings were not being cleared, causing infinite loops.
* Added job progress reporting in striker-manage-install-target, and made it only refresh the RPM repo when '--refresh' is specified (with --force now forcing the issue). This was done to allow adding it into anvil-daemon in such a way that it would only update the RPM repo once a day.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Get->anvil_version where the version of local systems and remote systems differed in closing new lines.
* Fixed a bug in Database->insert_or_update_variables() where the 'debug' parameter wasn't working.
* Renamed System->determine_host_type -> System->get_host_type.
* Fixed a bug in System->get_uptime where there was a newline after the uptime integer.
* Updated anvil-daemon to track and record the state of the Install Target feature on Striker dashboards.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-manage-install-target finished creating config files and enabling daemons needed for PXE. Still untested in function though.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Storage->read_file() where the last newline wasn't always being faithfully recorded.
* Created System->restart_daemon (as opposed to ->reload_daemon).
* Got creating/updating dhcpd.conf / dhcpd working in tools/anvil-manage-install-target.
Signed-off-by: Digimer <digimer@alteeve.ca>
* After much time wasted chasing a dnf bug (https://bugzilla.redhat.com/show_bug.cgi?id=1641947), tools/anvil-manage-install-target now populates the <DOCROOT>/<os_type>/<os_arch>/os/Packages/ directory with needed RPMs.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added 'sys::database::failed_connection_log_level' to allow silencing of log messages when a Striker peer database is not available.
* Started updating the .spec for the new release to add supported packages needed for PXE/dhcp/tftpboot.
* Added to repo tftpboot files as pulling them out of the packages and moving them into the right place relative to the modest size of adding them directly to our source wasn't justified.
* Created the still very very early 'tools/anvil-manage-firewall' tool.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added 'sys::log_date' which controls if the date and time is pre-pended to log entries.
* Created Get->host_name() which takes a host UUID and returns the 'host_name' from the 'hosts' table, if found.
* Cleaned up some HTML templates and logging.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the Database module to not sort or reorder the 'core_tables' array, and reordered them in the hash they're declared in.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added a check to all 'Database->insert_or_update_*' methods to check if the passed-in reference UUID was found and return an empty string if not.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated anvil-configure-striker to use Job methods and reboot using anvil-manage-power. Also updated it to set/clear maintenance mode and mark a reboot required at the end of it's run just prior to reboot.
* Lots of log cleanup.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Job->clear() to clear the job_picked_up_by column. Created Job->get_job_uuid() to return the job_uuid of an unfinished job matching a given job_command string (if any found).
* Updated striker->process_power to log the user out after confirming a poweroff or reboot action.
* Added anvil-daemon --startup-only to not enter the main loop and exit.
* Finished getting poweroff and reboot working (though more testing needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got the webui portion of requesting a poweroff and reboot done, but still working on finishing anvil-manage-power (work on which lead to the above improvement).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created the Anvil::Tools::Jobs module to handle general job processing task. Moved 'update_progress' from tools/anvil-update-system to it and generalized it.
* Added some missing CDATA wrappers to the words XML file strings with '>' in it.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Finished (for now) adding support for monitoring jobs while a node is in maintenance mode!
* Cleaned up the display of job data and redid how buttons (real and classed links) are displayed to be consistent.
* Fixed a bug in anvil-daemon where a disconnect wasn't being called between loops, causing DB connections to pile up.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created tools/anvil-clear-reboot to clear the "reboot needed" flag. Also created, but not yet using (and may not use) units/anvil-boot-time.service.
* Started work on having jobs show their data via JSON / jquery.
* Updated anvil-update-system to record messages indicating the progress so far.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Database->connect to take a specific UUID to attempt a connection to.
* Renamed some old 'sys::x' variables related to the database to 'sys::database::x' to conform better to coding standards.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated the RPM to .13 to disable postun's disabling of postgres, which breaks Anvil! software using the database during RPM updates.
* Fixed a logging bug where the number of DB connections was not inserting the number properly.
* Fixed exits in tools/anvil-prep-database.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-manage-striker-peers working properly (so far).
* Updated anvil-prep-database to call anvil-manage-striker-peers, but testing still needed.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Update Anvil::Tools->new() to access the parameters 'log_level', 'log_secure' and 'debug', streamlining the frequent calls to $anvil->Log->level and ->secure in program startup, and allowing the values to take effect during the ->new constructor.
* Passed 'debug' to child method calls in more places (still more to do though).
* Fixed a bug where 'test_table' wasn't set in the right place, causing the database to try to initialize repeatedly.
* Made Database->archive_database only run if called with root access.
* Now the number of database connections are stored in 'sys::db_connections' instead of checking the returned number, and that is cleared on disconnect.
* Started working more on 'anvil-daemon', including adding support for System->call being taking 'background', 'stderr_file' and 'stdout_file' paramters which, when set, used Proc::Simple to background the process.
* Did some more work on database archiving, though still far from done.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Set Log->entry to chmod the log file to 666 when the file is opened to ensure apache can write to it.
* Fixed a string replacement variable name.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated tools/anvil-prep-database to use a daemon name from Tools.pm to more easily support later postgres version upgrades.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where the old style '#!replace!...!#' replacement variables were not being escaped when processing variable insertions into strings.
* Made the body variable be stored in 'form::body' instead of passing around the '$body' variable.
* Created a set of new icons.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added error messages to Striker configuration forms.
* Fixed a bug in home->get_network_details() function to handle single IPs in network.xml.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug in Database->insert_or_update_network_interfaces where independent interfaces (not under a bridge or a bond) were not being saved.
* Continued working on improving Striker's network config jquery/form functions.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a typo in Database->insert_or_update_bridges().
* Added 'bond_mac_address' and 'bond_operational' to the bonds table.
* Gathered data needed for bonds in tools/anvil-update-states and got bonds recording successfully (but more testing needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added user_algorithm and user_hash_count to the new users database table so that we can remember how a hash was generated, should it be changed down the road.
* Made the salt length configurable by the user (as well as the algorithm and loop count).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created the new Account.pm module for handling account related tasks.
* Moved 'sys::stty' to 'sys::terminal:stty'.
* Created Get->_salt which generates a (pseudo) random salt for internal passwords and Get->_wrap_to that checks the current terminal width.
* Added the parameter 'redirect_stderr' (default '1') to System->call to optionally not redirect STDERR to STDOUT (needed for tput to be useful as STDERR is a terminal of sorts).
* Disabled System->change_apache_password (to be removed entirely later).
* Created Words->_wrap_string to wrap text strings to the active terminal width.
* Added the user's table to the core SQL. Also deleted the empty Anvil/Tools.sql file.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Made logging between journald and a traditional file configurable via 'sys::log_file'. Also made the file handle unbuffered when logging to a file.
* Fixed a bug with loading the anvil.conf config file in a few locations.
* Created System->stty_echo() to handle enabling/disabling shell echo, and added restoring the echo to Tools->catch_sig.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug with handling ssh fingerprints (and removed comments going to the known_hosts file).
* Added more nested debug parameter passing when methods call other methods (though more work is needed to catch up)
Signed-off-by: Digimer <digimer@alteeve.ca>
* Updated Storage->read_file and Storage->write_file to support reading and writing on remote systems (untested though)
* Created System->change_shell_user_password() that changes a shell user's password by manually generating an sha512 salted hash of the given password and uses the resulting hash to modify the target user's password, so the password should never be visible in the process list. Works on both local and remote systems, though it still needs testing.
* Created Storage->rsync() to handle moving files between the local and a remote system.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Fixed a bug where some '$anvil->{}' variables should have been '$anvil->data->{}'.
* Started merging message keys on 'error_xxxx', 'warning_xxxx', etc.
* The anvil-configure-network now configures the network. Commented out, the tool can reconfigure the entire network without a reboot, but a current issue with the post-configured system refusing to use the allocated interface as the default gateway is to be reviewed at a future time. For now, a closing reboot will be issued.
* Started creating 'anvil-change-password' that will update passwords, including apache (and configure .htpasswd when needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-configure-network writing out the new network config properly, but renaming already-active interfaces isn't working yet.
* Updated System->get_ips() to record the interface name of a given network by MAC address using 'sys::mac::<mac_address>::iface'.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got anvil-configure-network setting the new hostname.
* Updated anvil-configure-network to exit only if the job was picked up by a still-running PID.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Got tools/anvil-configure-network to exit if another instance is running. If not, it now starts (barely) the process of configuring the network.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Added Get->anvil_version() to check the local or remote Anvil! version.
* Added a check in Database->connect() to see if a database server's Anvil! version matches the local version. If the versions don't match, the database is not used.
Signed-off-by: Digimer <digimer@alteeve.ca>
* Moved all executables to /usr/sbin/
* Made /root/anvil-backups/ the backup directory.
* Started debuging anvil-prep-database
Signed-off-by: Digimer <digimer@alteeve.ca>
* Created Storage->record_md5sums() and Storage->check_md5sums for use in daemons. These will record the md5sums of the program itself, all perl modules and the words file. When check_md5sums is called, it returns '1' if any sums have changed, which daemons can trigger on to exit (and systemd will restart them). Removed the basic md5sum check from anvil-daemon and switched to this.
* Fixed how 'fatalstobrowsers' is invoked so that it only applies to programs running in a browser.
Signed-off-by: Digimer <digimer@alteeve.ca>
We've decided to give up on trying to keep ScanCore, AN::Tools and Striker as three separate things. We had originally hoped to make ScanCore easily separatable from the Anvil!, but this was adding increasing complexity to the project and complexity is the enemy of reliability.
In this release, AN::Tools becomes Anvil::Tools, all configuration files move to /etc/anvil and all programs and data files move to /usr/sbin/anvil. Words files are now merged, as are SQL schemas (ScanCore agents will still maintain their own, later). The journald tag has changed from 'an-tools' to 'anvil'.
Other changes;
* Tools.t has been updated to handle existing tests. New methods and parameters still need to have tests added though.
* Added a simple test.pl script used for testing things outside the main program. It will be removed before final release.
* Added the simple 'watch_logs' bash script to more easily tail output.
Signed-off-by: Digimer <digimer@alteeve.ca>
