Commit Graph

127 Commits

Author SHA1 Message Date
Digimer
7f5ac528ab * Got tools/anvil-manage-install-target to now properly build the install repo source! Installing guests is now working, at a base level. Still need to add Anvil! specific stuff, and firewall support. Also decided to emulate the upstream 'Packages' directory structure and moved all packages in to subdirectories based on the first letter of the package.
* Bumped the default PXE menu timeout to 10 minutes.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-11-05 03:33:43 -05:00
Digimer
ea65fa08aa * Added generation of kickstart files from a single template and made them more configurable by the user via anvil.conf.
* Made the PXE spash image come from the active skin.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-11-03 02:55:35 -04:00
Digimer
b815af03df * Created a new RPM (different repo; anvil-striker-extra) for files needed for the Install Target feature but not available in (or reasonably extracted from) RPMs.
* More work on anvil-manage-install-target

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-11-02 02:06:04 -04:00
Digimer
45f290d5e7 * Updated all System->*_daemon methods to not append '.service' to the daemon parameter. Also make all daemon names variables stored in 'sys::daemon::<foo>' and updated existing calls to these methods to use those variables.
* Got anvil-manage-install-target finished creating config files and enabling daemons needed for PXE. Still untested in function though.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-11-01 03:16:07 -04:00
Digimer
6f5541b6fd * Created Get->network() that returns the base network address for a given IP/subnet.
* Fixed a bug in Storage->read_file() where the last newline wasn't always being faithfully recorded.
* Created System->restart_daemon (as opposed to ->reload_daemon).
* Got creating/updating dhcpd.conf / dhcpd working in tools/anvil-manage-install-target.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-31 01:42:24 -04:00
Digimer
380b11ce55 * Created Storage->update_file() that will update a file _if_ the contents have changed, locally or remotely and log the diff of the changes that triggered the update.
* Got tools/anvil-manage-install-target to generate and record the PXE BIOS 'default' file from template.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-29 19:50:43 -04:00
Digimer
706052105e * Changed Storage->read_file() to default 'force_read' to '1', causing it to always read from disk and ignore cache unless set to '0'.
* After much time wasted chasing a dnf bug (https://bugzilla.redhat.com/show_bug.cgi?id=1641947), tools/anvil-manage-install-target now populates the <DOCROOT>/<os_type>/<os_arch>/os/Packages/ directory with needed RPMs.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-24 02:21:41 -04:00
Digimer
3045deb09d * Created tools/anvil-manage-install-target to setup a Striker machine as an install target. Still early in development.
* Created System->get_os_type() to get a short form of the host system OS and version.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-23 12:55:19 -04:00
Digimer
702cb0f102 * Almost finished the first draft of the 'plan_partitions' kickstart (sub)script.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-12 03:28:42 -04:00
Digimer
07c3b405ad * Starting work on adding "Install Target" function (will likely rename this, but basic same function as IT in m2).
* Added 'sys::database::failed_connection_log_level' to allow silencing of log messages when a Striker peer database is not available.
* Started updating the .spec for the new release to add supported packages needed for PXE/dhcp/tftpboot.
* Added to repo tftpboot files as pulling them out of the packages and moving them into the right place relative to the modest size of adding them directly to our source wasn't justified.
* Created the still very very early 'tools/anvil-manage-firewall' tool.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-10 19:43:23 -04:00
Digimer
9a4cefe3fb * Fixed a bug (which is really more of a work-around for a deeper bug to be addressed later) where the database would keep thinking it needed to resync, even after it had successfully.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-07 02:56:55 -04:00
Digimer
6f0bc0d86f * Fixed a major bug where anvil-daemon would reset the job_progress to 0 when clearing the 'reboot_needed' flag, causing anvil-daemon to pick the job up and again reboot, repeatedly.
* Updated Jobs->update_progress to take 'picked_up_by' as an optional parameter, defaulting to '$$' (the caller's PID).
* Created System->get_uptime() to return the current uptime in seconds.
* Added a delay to anvil-manage-power to not proceed with a reboot if the uptime is less than 600 seconds. This way, if any future bug causes an infinite reboot, there will be more time to determine what's wrong and debug the system between reboots.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-06 03:16:08 -04:00
Digimer
510321d634 * It looks like adding and removing Striker peers (and all the sync'ing stuff behind it) is finally sorted out. Obviously, time will tell for sure, but currently things look good.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-04 20:54:00 -04:00
Digimer
43035ba038 * Fixed a subtle an annoying autovivication bug in Database->write().
* Cleaned up some logging.
* Made the "Reload" buttons work more sensibly and cleaned up some webui display stuff.
* Got deleting peers mostly working (well, it works, but then it goes into a loop thinking it needs to resync the now-gone database until the daemon restarts).
* Fixed a race condition bug where if a job exited between the time that anvil-daemon got a list of PIDs and when it checked to see if that specific pid was alive, a job that actually completed could be restarted.
* Added a loop check to anvil-manage-striker-peers where it would hold until a database connection to the newly added peer was available, preventing a condition where re-adding a peer (and so the host_uuid is in hosts) cause the job belonging to the peer to be recorded locally and then never synced to the peer.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-04 03:37:43 -04:00
Digimer
e79e7fd4f4 * Added 'check_if_configured' to Database->connect(), disabled off, that triggers the check to see if the system is configured or not. Updated anvil-daemon to invoke this at the same time that the md5sums are calculated to see if a reload is needed. This reduces the background system load a fair bit.
* Got more work done on deleting peers from Striker (technically done, but untested so far).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-10-02 03:31:42 -04:00
Digimer
facefeaccc * Fixed a bug in anvil-daemon where completed jobs could be immediately cleared, causing them to re-run (repeatedly).
* Added 'sys::log_date' which controls if the date and time is pre-pended to log entries.
* Created Get->host_name() which takes a host UUID and returns the 'host_name' from the 'hosts' table, if found.
* Cleaned up some HTML templates and logging.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-28 01:50:38 -04:00
Digimer
d3d28d8ec5 * Updated striker's first config page to pre-fill with values (Org name, prefix, sequence and domain name) if they can be devined from the existing system config.
* Updated System->hostname() to return the "pretty" host name as well as the traditional host name.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-26 18:55:52 -04:00
Digimer
f58979ad27 * Fixed a bug in Jobs->clear() where the 'job_uuid' wasn't being passed to Database->insert_or_update_jobs().
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 16:28:35 -04:00
Digimer
90bbcfe9f4 * Updated Database->insert_or_update_jobs() to have 'update_progress_only' conditionally update 'job_status', 'job_picked_up_by', 'job_picked_up_at' and 'job_data'.
* Updated Database methods that can take the caller's file or line to append the local file and line.
* Updated Jobs methods to call Database->insert_or_update_jobs().
* Updated the remaining UPDATEs of 'jobs' table to use 'Database->refresh_timestamp()' for the 'modified_date'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 13:50:48 -04:00
Digimer
bbf194994e * New attempt to solve the resync issue; Created a new method called Database->refresh_timestamp that will update 'sys::database::timestamp', and call it when UPDATE'ing in '->insert_or_update_jobs()'. This should force the 'modified_date' to be unique per update and allow the existing resync approach to work properly. If so, this method will be used later for any instance where a given record can be updated multiple times per run.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 02:15:44 -04:00
Digimer
9bd5dd9a18 Revert to bfc2204.
This reverts commit bfc2204352.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 02:05:07 -04:00
Digimer
950338da6c * More work on Database->_split_query(), though I may be about to revert as this is proving exceedingly complicated and I may have a simpler solution.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-25 01:43:12 -04:00
Digimer
e07bb0908d * Did more work on parsing queries in Database->_split_query(), but still quite a lot to do.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-24 13:59:21 -04:00
Digimer
a8369170b4 This is the start of a major change!
The resync of the databases was originally designed (on m2) with the expextation that any given column would have only one change per 'modified_date' time. That was never a great approach, but it worked in m2 and just bit me on m3. With job processing, for an example, the job_progress will change repeatedly in one pass, all with the same 'modified_date'. So only one record per run would resync. To fix this, the plan is to drop 'history_id' (and the procedure/trigger in pgsql to copy INSERT and UPDATEs to the history schema). The new plan is to use 'change_uuid' with a per-transaction UUID created in Database so that the per-DB 'history_id' is replaced with a per-update/insert UUID in 'change_uuid'. This will become the unique record used to sync databases, instead or 'modified_date'. To keep things consistent, 'modified_date' was renamed to 'change_date' to match 'change_uuid'. This work is very much "in progress" and not finished.

This commit also changes Get->uuid to use UUID::Tiny to create v4 UUIDs instead of making making a system call to 'uuidgen'. This sped up UUID generation by almost 100x.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-23 16:16:08 -04:00
Digimer
bfc2204352 * Added a row-count check when deciding if a DB resync is needed.
* Updated the Database module to not sort or reorder the 'core_tables' array, and reordered them in the hash they're declared in.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-22 00:54:37 -04:00
Digimer
40aac1d5f6 * Finished adding the 'sessions' database table and associated code.
* Added a check to all 'Database->insert_or_update_*' methods to check if the passed-in reference UUID was found and return an empty string if not.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-21 13:49:28 -04:00
Digimer
b2367bb90b * Started working on moving user sessions to the new 'sessions' table. This is still a work in progress.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-17 15:34:43 -04:00
Digimer
e483d6e9a1 * Updated Database->connect() to delete 'database::<uuid>' when a peer's database is not a matching version to avoid later use attempts.
* Fixed a bug in Database->resync_databases() where tables that pluralized with the suffix 'es' were not being singularized properly, so the '_uuid' column wasn't found and preventing resync of that table.
* Fixed a bug in System->maintenance_mode() and System->reboot_needed() where the variable read wasn't restricted to the local host UUID.
* Reordered the config and maintenance mode checks in striker so that configure is shown even if maintenance mode is somehow set without configuration being done.
* Updated the .spec for .16 RPM release.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-14 02:23:33 -04:00
Digimer
c6e781d80b * Adding database peers (local and bidirectional) now works!
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-13 17:08:50 -04:00
Digimer
7bd65f65e5 * Finished the first round of updates to anvil-manage-striker-peers, but the initial resync is failing because of unrelated schema issues.
* Updated Database->insert_or_update_jobs() to also use the job_command when looking for an existing job (when a specific job_uuid was not included).
* Fixed a bug with a missing ? in striker->add_sync_peer function. Also updated it to not try to record the peer's job as it is unlikely the peer will be in hosts. Instead, the job_command to add the peer is appended to the local job't job_data and the updated anvil-manage-striker-peers looks for that at the end of the add and sync, and records the job once the peer's UUID is in 'hosts'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-13 02:22:35 -04:00
Digimer
2a03663ecf * Created System->find_matching_ip() which takes a host and finds which local network (if any) is on the same subnet.
* Continued work on managing Striker peers.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-10 19:54:22 -04:00
Digimer
39c94009e6 * Created System->check_if_configured and then used that to only have anvil-daemon call update_state_file() when the system is unconfigured (to reduce the laod) when it's usually not needed).
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-07 22:55:05 -04:00
Digimer
94d8a9c495 * Fixed a bug where finished jobs with a '0' picked-up time would be written to jobs.json.
* Updated anvil-configure-striker to use Job methods and reboot using anvil-manage-power. Also updated it to set/clear maintenance mode and mark a reboot required at the end of it's run just prior to reboot.
* Lots of log cleanup.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-07 01:29:43 -04:00
Digimer
00565b123c * Updates Tools->nice_exit to add the caller name to the exit status.
* Created Job->clear() to clear the job_picked_up_by column. Created Job->get_job_uuid() to return the job_uuid of an unfinished job matching a given job_command string (if any found).
* Updated striker->process_power to log the user out after confirming a poweroff or reboot action.
* Added anvil-daemon --startup-only to not enter the main loop and exit.
* Finished getting poweroff and reboot working (though more testing needed).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-06 01:37:08 -04:00
Digimer
b7e4ba9123 * Made the detection of whether a system has been rebooted a lot smarter, thanks to an idea from Lisa Seelye (@thedoh).
* Got the webui portion of requesting a poweroff and reboot done, but still working on finishing anvil-manage-power (work on which lead to the above improvement).

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-05 03:39:13 -04:00
Digimer
545f9a9bb5 * Renamed tools/anvil-reboot-needed to tools/anvil-manage-power and started adding support for rebooting and powering off to it.
* Created the Anvil::Tools::Jobs module to handle general job processing task. Moved 'update_progress' from tools/anvil-update-system to it and generalized it.
* Added some missing CDATA wrappers to the words XML file strings with '>' in it.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-04 18:57:09 -04:00
Digimer
962ff89fc5 * Fixed a bug in Words->parse_banged_string() where values with commas was breaking the processing of the string of variable/value pairs.
* Added '--refresh-json' to anvil-daemon that auto-selects '--run-once', '--main-loop-only' and '--no-start'.
* Updated anvil-update-system to not go more than a second between updates to the progress (save for when we're holding on data from 'dnf').

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-09-03 02:35:25 -04:00
Digimer
15994727fb * Quick space fix
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-31 03:00:27 -04:00
Digimer
423fda2ad6 * Fixed a bug in anvil-daemon where rebooting was clearing the reboot-needed flag.
* Finished (for now) adding support for monitoring jobs while a node is in maintenance mode!
* Cleaned up the display of job data and redid how buttons (real and classed links) are displayed to be consistent.
* Fixed a bug in anvil-daemon where a disconnect wasn't being called between loops, causing DB connections to pile up.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-31 02:40:49 -04:00
Digimer
eaca4c885f * Updated tools/anvil-daemon to use the JSON module to build the JSON strings going into jobs.json, instead of doing it manually.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-29 00:33:33 -04:00
Digimer
831ff14d93 * Created Words->parse_banged_string to process strings in the format '<key>[,!!var1!value1!!,!!var2!value2!!,...,!!varN!valueN!!'. Still testing this.
* Made anvil-update-system look for a job_uuid when none is passed.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-28 22:42:46 -04:00
Digimer
60584b8cee * Created Database->get_jobs() to be a more general way to retrieve pending and recently finished jobs.
* Started adding the display of running and recently finished jobs to Striker when in maintenance mode. Still lots to do.
* Started working on the logic for what will soon be Words->decypher_string in anvil-daemon to process strings stored as '<key>,!!<name1>!<value1>!!,...,!!<nameN>!<valueN>!!'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-28 01:54:49 -04:00
Digimer
bd862b2e5e * Made more progress on anvil-daemon's invokation of jobs.
* Got anvil-update-system clearing job data when (re)starting

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-27 02:51:53 -04:00
Digimer
633da25d07 * Fixed a bug in Database->connect where an empty 'database::<key>' would cause an error. Updated Database->disconnect to delete the 'database' hash key as part of the same fix.
* Renamed 'database::locking' to 'sys::database::locking' to avoid collisions with 'database' keys.
* Fixed a problem with System->call where reidrects were missing the Proc::Simple method name.
* Updated anvil-daemon to check if there is no database connections on start-up, run prep-database if not, and try connecting again. If it still fails, exit. Also updated the main loop to reconnect to the database(s) and skip if non are available. Did more work on the keep_running() function.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-24 00:52:56 -04:00
Digimer
18455ab5f7 * Created System->reboot_needed() to set/clear/report if a reboot is needed.
* Renamed tools/anvil-clear-reboot to tools/anvil-reboot-needed and changed it to behave like anvil-maintenance-mode.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-22 19:50:20 -04:00
Digimer
0191f93bad * Created tools/anvil-maintenance-mode to enable/disable maintenance mode from the command line.
* Fixed a bug in System->maintenance_mode where disabling it wouldn't work.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-22 17:57:50 -04:00
Digimer
d147e10552 * Changed the call to reboot after OS update to instead set a reboot required flag.
* Created tools/anvil-clear-reboot to clear the "reboot needed" flag. Also created, but not yet using (and may not use) units/anvil-boot-time.service.
* Started work on having jobs show their data via JSON / jquery.
* Updated anvil-update-system to record messages indicating the progress so far.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-22 02:16:56 -04:00
Digimer
51c9d2952d * Fixed a bug with caching database handles in the recent database updates causing Database->write to stop working.
Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-16 19:34:37 -04:00
Digimer
fe33fbb239 * Created System->maintenance_mode to add "maintenance mode" support to Striker.
* Continuing work on adding the system update job support to Striker. Untested/Incomplete.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-16 02:38:58 -04:00
Digimer
6aa74d3d96 * Updated Database->_test_access() to use the DBD 'ping' method, and attempt a reconnect of failure.
* Updated Database->connect to take a specific UUID to attempt a connection to.
* Renamed some old 'sys::x' variables related to the database to 'sys::database::x' to conform better to coding standards.

Signed-off-by: Digimer <digimer@alteeve.ca>
2018-08-15 16:57:57 -04:00