Commit Graph

527 Commits

Author SHA1 Message Date
Digimer
9bcad1c4bf * Did more work on adapting tools/anvil-configure-host to work on nodes and DR hosts.
* Fixed a bug in Job->get_job_details() when no job_uuid was passed.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-11 00:38:48 -05:00
Digimer
b97f15b25e * Finished the sanity checks and user confirmation table when preparing to configure the network on a node or dr host.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-10 02:38:55 -05:00
Digimer
6d81e03fb2 * Created Network->match_gateway() to check if a gateway applies to a given network.
* Got more work done on confirming the user's request to setup the network of a node or DR host.
* Reworked network select boxes to sort by the network name instead of the MAC address.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-08 00:19:13 -05:00
Digimer
86af67ecda * Created a cachine mechanism for anvil-update-states so that it can record network interface link state changes when it loses contact with all databases, as can happen when cycling NICs to map a newly build DR host or node.
* Updated Database->insert_or_update_network_interfaces() to take the new 'link_only' and 'timestamp' parameters to support flushing out the cache file above.
* Updated anvil-daemon to run anvil-update-states when the database connection is lost. Also moved the 'handle_periodic_tasks()' function call to be conditional on there being a database connection.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-05 01:59:41 -05:00
Digimer
530d379f59 * Started work on caching network state change in tools/anvil-update-states.
* Fixed a bug where ip_addresses could break resync when 2+ machines had the same IP (ie: 192.168.122.1).
* Updated logging of DB transactions to show the DB host's IP instead of the UUID.
* Updated Get->date_and_time to take a 'use_utc' parameter to return the time using GMT time instead of the host's TZ.
* Updated anvil-daemon to periodically call tools/anvil-update-states. Also upadted anvil-daemon to delay daily jobs by 2 hours except for the dashboard with the highest sorted UUID to minimize dual runs of tasks that only need to run once per day per cluster.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-04 00:02:19 -05:00
Digimer
e8d15112da * Fixed a bug in anvil.js where the state of a link always said 'up', even when it was down.
* Fixed a couple logging bugs in System->call().
* Fixed a bug in anvil-daemon where it was trying to setup setuid-C wrappers on non-dashboards.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-03 01:23:31 -05:00
Digimer
90f5bf49d5 * Updated Network->load_interfces() to only assign a 'changed_order' to real interfaces.
* Fixed a bug in System->generate_state_json() where interfaces connected to a bridge were constantly having their 'network_interface_bond_uuid' cleared and reset.
* Finished (for now) the jquery code to update the network interface list when preparing the network interface configuration of a new node or DR host.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-03 00:30:26 -05:00
Digimer
0dfd2ddbf0 * Renamed 'ip' to 'ip_address' in Striker->parse_all_status_json() and related functions.
* Got the menu for mapping a host's network displaying (much work still to be done).
* Updated the anvil.js funtion to run dependent on the page being shown. For the main menu, the json is now properly reread and display updated as json content changes.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-29 23:37:11 -05:00
Digimer
0b2439edc3 * Finished the jquery/css stuff for showing the list of hosts that can have their network configured.
* Fixed unclear error logging in Network->find_matches().
* Updated System->generate_state_json() Striker->parse_all_status_json() to determine if another machine can be reached from the local dashboard. If it can be reached, the first matching interface and IP are recorded.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-27 23:26:21 -05:00
Digimer
c608ebb232 * Added bonds -> bond_bridge_uuid to track which bridge, if any, that a bond is connected to. Updated Network->load_interfces() to record this.
* Added missing foreign key references to the SQL schema.
* Added support to tools/anvil-update-states to connect bonds to bridges, as appropriate.
* Finished the logic in test.pl to pull the network data (with connections between bridges, bonds and interfaces) needed for the WebUI.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-26 00:15:37 -05:00
Digimer
0cf13b6387 * Worked out more of the logic in test.pl for displaying the JSON data.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-22 18:14:25 -05:00
Digimer
82bc25c78d * Adding the test.pl script. Even though it doesn't relate to the program, it's used to sort out logic and the code should be committed, until mainline development is done.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 19:47:21 -05:00
Digimer
7aaef3d6d5 * Fixed bugs in Striker->parse_all_status_json() and System->generate_state_json() where the connected interfaces to bridges and bonds wasn't be stored properly.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 19:23:16 -05:00
Digimer
aba79cc457 * Added a check in Striker->parse_all_status_json() to set the connected interface types.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 14:07:03 -05:00
Digimer
54de5466cf * Renamed bond_active_slave to bond_active_interface.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 10:55:21 -05:00
Digimer
12b0cc28f0 * Renamed 'primary_slave' to 'primary_interface' for the bond table.
* Got more work done on parsing the all_status.json (and fixing related bugs).

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-20 01:45:16 -05:00
Digimer
d5672e77eb * Standardized 'subnet_mask' over 'subnet' naming throughout.
* Created (and haven't finished yet) Striker->parse_all_status_json().

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-19 02:13:19 -05:00
Digimer
b2ff4c4c53 * Created Network->get_company_from_mac() that converts a MAC address to the company name that owns it.
* Added recording the last-change order for network interfaces in System->generate_state_json() so that the most recently unplugged and plugged back in interfaces can be tracked.
* Worked out a faster way to ping scan subnets with nmap in striker-scan-network. Dropped average scan time from 35 minutes to 4~5 minutes for a /16 subnet.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-15 18:29:07 -05:00
Digimer
e3a8c1a01d * Created System->generate_state_json() that reads, parse and writes out the network status of all known machines on a given Striker database.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-14 23:58:39 -05:00
Digimer
387c03aa7d * Added more text for the pending JSON state file generation.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-13 21:29:51 -05:00
Digimer
7cf1db3b37 * Added strings for making size suffixes translatable and updated Convert->bytes_to_human_readable to use them.
* Fixed a bug in tools/anvil-update-states where the MAC of an interface that is the backup in an active-backup bond would be the MAC of the active member instead of its real MAC.
* Fixed a bug in Convert->add_commas() where a passed in value of '0' returned an empty string.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-12 16:27:01 -05:00
Digimer
b9e8d2a971 * Added suffixes for data sizes.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-11 18:18:29 -05:00
Digimer
d0006cc4d3 * Changed a bunch more variables to standardize on 'mac_address' instead of 'mac'.
* Added ' --best --allowerasing' to the dnf call in tools/anvil-update-system (found it was needed when 'yum-utils' replaced 'dnf-utils' in RHEL 8.1 release).

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-11 11:17:02 -05:00
Digimer
70776964d6 * Added packages needed to support offline machines to striker-manage-install-target.
* Cleaned up the returned 'return_code' in System->call() when background is true.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-08 14:49:49 -05:00
Digimer
ea27f88bcc * Removed the 'bridge_interfaces' table and related database methods. This was found to be duplicate, given that 'network_interfaces' has references to parent bonds and bridges.
* Renamed 'bridges' -> 'bridge_mac' to 'bridge_mac_address' to be consstent with other MAC address column names.
* Finished Network->load_interfces().
* Updated anvil-update-states to check for interfaces under bridges that are missing their 'network_interface_bridge_uuid' reference UUID.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-07 23:40:02 -05:00
Digimer
628f7faa45 * Updated the RPM spec file to generate '/etc/anvil/type.X' files to directly indicate the machine type. Updated System->get_host_type() to check for these files directly, falling back to parsing the host name if they don't exist.
* Created Database->get_hosts_info() (though it's not at all finished) that will write out a unified JSON file contain all data known about all hosts/Anvil! systems. This will be later used to create the WebUI parts.
* Also created, but also not finished, Network->load_interfces() that will work sort of like ->load_ups, but include all interfaces regardless of if they have an IP or not.
* Fixed a bug where the new bridge_interface_note parameter didn't exist in the Database->insert_or_update_bridge_interfaces() method.
* Updated anvil-update-states() to only write out the JSON/XML files if it's running on a dashboard. For nodes and DR hosts, it just needs to update the database.
* Created a new hook in anvil-daemon that will call tasks on a machine that is configured.
* As per RHEL 8.1 release notes, changed the package 'dnf-utils' to 'yum-utils' in the packages to load for install target repos.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-06 11:26:21 -05:00
Digimer
077977ad9c * Fixed tools/anvil-update-states so that it properly removed ip_addresses no longer assigned to a host. Also merged 'network::interfaces::by_name::${interface}' with 'network::local::interface::${interface}' for storing discovered interfaces.'
* Added 'ip_address_note' to the 'ip_addresses' table as there was no column convenient for flagging as DELETEd.
* Added 'uuid' to Database->insert_or_update_file_locations() and ->insert_or_update_files(), and actually used it in all ->inser_or_update_X() methods.
* Added 'delete' as a parameter to Database->insert_or_update_ip_addresses() to allow simple deletion of a referenced IP address.
* Addressed a few 'undefined variable' errors.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-02 00:02:36 -04:00
Digimer
517771594e * Updated to release -28 RPM.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-28 11:04:57 -04:00
Digimer
197f5ed90e * Finished the automatic archving of the database! New in M3; Archving happens on dashboards only, and whichever database triggers an archive will archive all data in any connected databases. This is a departure from where each host archived it's own data, but will avoid putting a disk or CPU load on server hosts.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-28 02:27:26 -04:00
Digimer
650dc6eb80 * Finished Storage->compress(). Also added some missing 'port' parameters (reads and passes) in a few different Storage methods.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-27 17:19:03 -04:00
Digimer
deb8965a38 Merge branch 'master' of github.com:digimer/anvil
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 20:43:17 -04:00
Digimer
08176a5711 * Updated Database->_archive_table() to remove double-// in paths.
* Started work on Database->compress.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 20:10:02 -04:00
Digimer
b29bfd4c59 * Added some logging to the archive method.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 20:07:03 -04:00
Digimer
b436500a54 * Started working on getting database archiving working. It's close to done, but needs testing/debugging and other finery.
* Created Database->get_host_from_uuid() that takes a host UUID and returns the host's name.
* Reworked Network->find_matches() to return both the match's IP and subnet.
* Finished getting tools/striker-initialize-host to add all known peers to the target, using IPs on the target's available subnet.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 01:53:52 -04:00
Digimer
b1a175baa0 * Finished Network->load_ips().
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-23 11:10:07 -04:00
Digimer
934c9b1286 * Updated logging to now log anything with 'priority' set to a new 'anvil.alert.log' file (while still also logging as normal to anvil.log). This should make it easier to watch for alert messages.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-22 21:17:30 -04:00
Digimer
f750e087f6 * Prepped .spec for -27 release.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-20 23:09:17 -04:00
Digimer
af6e2c076d * Fixed a tricky deep recursion bug in Network->is_local when the passed in host was an empty string. Also created a cache system where a host name that has been checked before is immediately returned, without needing to run through the logic in 'is_local', which gets called quite frequently.
* Updated the loop detection logic in Log->entry where processing large strings was triggering it when it shouldn't.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-20 22:55:49 -04:00
Digimer
32bcdbe6d3 * Removed Network->is_remote, standardized on Network->is_local, and flipped calls to it to be more sensible (is_local -> local call -> else remote call). Also fixed a deep recursion issue with ->is_local where, given that it logs (which calls Storage methods which have local/remote invocations), would loop.
* Fixed a bug where '$target' being preset to 'local' was causing bad calls to 'Remote->call'.
* Updated Storage->change_mode and -> change_owner to work locally and on remote hosts.
* Barely started work on striker->process_anvil_menu().

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-19 00:57:33 -04:00
Digimer
a7f93c59ea * Reworked striker-parse-oui and striker-scan-network to always lower-case the MAC address.
* Updated striker-scan-network to only run once per day unless --force or a given --network is used. This avoids repeated scans when the anvil-daemon restarts frequently for whatever reason.
* Fixed (for real this time) Convert->time's handling of the 'long' parameter.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-18 11:40:38 -04:00
Digimer
4d0a02ce74 * Fixed a bug in Database->get_local_uuid() where ->is_local() was being called incorrectly.
* Added job parsing to tools/striker-parse-oui and tools/striker-scan-network, and enabled them in anvil-daemon.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-17 13:08:11 -04:00
Digimer
1d13e669a7 * Created the new tools/striker-scan-network tool that ping scans a network range and records the discovered hosts in the new 'mac_to_ip' table. Also created Database->insert_or_update_mac_to_ip() to handle the new table.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-16 23:23:02 -04:00
Digimer
7e960f1632 * Created the 'oui' database table that stores the parsed OUI data, as processed by the new 'tools/striker-parse-oui' tool. Also created Database->insert_or_update_oui() to handle inserts and updates.
* Fixed a bug in Convert->time() where the suffix was long when it should have been short, and vice-versa.
* Updated Network->download() to check if the target file exists and, if so, to abort unless 'overwrite' is given or the existing file is 0-bytes long. Also updated it to not exit on immediate error after the wget call and instead check to see if a zero-byte file exists and remove it, if so.
* Created Validate->is_hex() to check hexadecimal strings.
* Updated Words->clean_spaces() to remove MS-DOS-style ^M cr/lf characters.
* Updated anvil-daemon to have a section for periodic tasks that run daily, and added striker-parse-oui as well as moved striker-manage-install-target refresh to that check. Also made those tools run on dashboards only.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-16 01:57:59 -04:00
Digimer
7cdd2f60e9 * Created Network->download() to handle downloading a file on the local system. Created ->bridge_info() to parse 'bridge' output. Created ->load_ips() to load IP address information from the database (as opposed to ->get_ips() which queries a system).
* Created Database->insert_or_update_bridge_interfaces() to handle updating the new bridge_interfaces database table that records which network interfaces are connected to which bridges.
* Added 'bridge_mac' and 'bridge_mtu' to the 'bridges' table.
* Started Server->map_network which will, eventually, try to map MAC addresses to IPs and record server -> vnetX -> bridge data. Made getting a server status target-dependent.
* Worked on anvil-update-states to parse and record bridge data.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-12 00:32:32 -04:00
Digimer
183d2d9cce * Updated Storage->backup to append a short UUID to the timestamp to prevent issues if the same file is backed up twice in the same clock second. Also fixed a bug with remote_user parameter not having a default.
* Finished the detection of and handling of initialization of a host when the host has no Internet access.
* Disabled (for now) anvil-daemon's check_ssh_keys function.
* Fixed a couple small bugs elsewhere.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-04 22:36:39 -04:00
Digimer
65a27ba2f8 * Created Network->check_internet() that, well, checks to see if a machine has access to the Internet or not.
* Moved System->is_local to Network->is_local, and System->ping to Network->ping.
* Added a check to tools/striker-get-peer-data that will report if the target has Internet access or not.
* Cleaned up the form that prompts the user to enter their Red Hat credentials.
* Updated tools/anvil-manage-keys (and related code) to no longer distinguish by user. If a target is flagged as changed, it is removed from the root and all user's known_hosts files.
* Updated Storage->write_file() and ->update_file() to accept the 'backup' parameter to control if an file that exists is backed up before being updated/replaced.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-04 00:40:30 -04:00
Digimer
f65a2897a7 * Prepped v3.0.26 RPM release
* Fixed minor bugs in tools/anvil-configure-host, tools/anvil-manage-files, and tools/striker-get-peer-data

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-02 17:23:03 -04:00
Digimer
3a86bed694 * Fixed tools/striker-initialize-host so that it set the hostname on the target, not locally.
* Updated System->host_name to work locally and on remote targets.
* Renamed all 'hostname' instances to 'host_name' to standardize on a spelling throughout the program.
* Removed use of and dependency on 'hostname'.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-02 15:39:21 -04:00
Digimer
b57546e663 * Fixed a bug in Database->manage_anvil_conf() where the wrong password and port was being set.
* Fixed a bug in Get->host_uuid() where the call to get the host UUID from dmidecode was broken.
* Updated striker -> Initialize host to allow the user to set the host name of a node or host being initialized, allowing it to be registered with Red Hat under the proper name and make it easier to track which machine is which during initial Anvil! build.
* Fixed a few minor bugs with variable insertions into translated strings.
* Updated striker-initialize-host to use a progressive progress value rather than statically assigned steps.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-02 02:04:22 -04:00
Digimer
3e01537209 * Updated how bad keys are removed. No longer are speciifc lines removed by number, but now all entries for the given target in the known_hosts file is removed.
* Updated the pxe.txt file to now write a caller for anvil-update-issue in /etc/NetworkManager/dispatcher.d/ifup-local to have the /etc/issue file is updated as soon as the network is brought up, before the GDM login prompt is shown.
* Fixed a couple bugs in tools/anvil-manage-keys, including to ensure that the permissions are retained when a file is updated.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-09-27 23:21:24 -04:00