Commit Graph

582 Commits

Author SHA1 Message Date
Digimer
c34f56f2c8 * Disabled firewalld for now.
* Created System->disable_daemon.
* Fixed a bug in Database->insert_or_update_bonds() where the bridge_uuid was an empty string instead of NULL.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-17 01:38:42 -05:00
Digimer
b4a979da4b * Bumped the rpm to -30.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-13 23:54:49 -05:00
Digimer
a16e86dc7c * Finished tools/anvil-configure-host! More notably, it can now update the network without a reboot, and it cleanly handles losing and reconnecting to dashboards.
* Added 'no_ping' to Database->connect() to disable pinging before connection, regardless of the anvil.conf setting.
* Created Network->read_nmcli() that reads, parses and stores the verbose output from 'nmcli'.

I can not properly explain in this commit message how much getting network manager working tripped me up. omg the complexity of what used to be such a simple process...

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-13 22:20:18 -05:00
Digimer
414bf81f25 * Fixed a bug where the BOOTPROTO="x" was set to dhcp for a bond interface that is connected to a bridge and should have been set to 'none'.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-11 01:25:18 -05:00
Digimer
e9ee74553a * Fixed a comment bug in Network->reconfigure_network() indicating that MAC lookup was using the wrong hash key.
* Fixed a couple more bugs in tools/anvil-configure-host, getting it now to the point where it writes out the network config files.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-11 01:03:50 -05:00
Digimer
9bcad1c4bf * Did more work on adapting tools/anvil-configure-host to work on nodes and DR hosts.
* Fixed a bug in Job->get_job_details() when no job_uuid was passed.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-11 00:38:48 -05:00
Digimer
b97f15b25e * Finished the sanity checks and user confirmation table when preparing to configure the network on a node or dr host.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-10 02:38:55 -05:00
Digimer
6d81e03fb2 * Created Network->match_gateway() to check if a gateway applies to a given network.
* Got more work done on confirming the user's request to setup the network of a node or DR host.
* Reworked network select boxes to sort by the network name instead of the MAC address.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-08 00:19:13 -05:00
Digimer
86af67ecda * Created a cachine mechanism for anvil-update-states so that it can record network interface link state changes when it loses contact with all databases, as can happen when cycling NICs to map a newly build DR host or node.
* Updated Database->insert_or_update_network_interfaces() to take the new 'link_only' and 'timestamp' parameters to support flushing out the cache file above.
* Updated anvil-daemon to run anvil-update-states when the database connection is lost. Also moved the 'handle_periodic_tasks()' function call to be conditional on there being a database connection.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-05 01:59:41 -05:00
Digimer
530d379f59 * Started work on caching network state change in tools/anvil-update-states.
* Fixed a bug where ip_addresses could break resync when 2+ machines had the same IP (ie: 192.168.122.1).
* Updated logging of DB transactions to show the DB host's IP instead of the UUID.
* Updated Get->date_and_time to take a 'use_utc' parameter to return the time using GMT time instead of the host's TZ.
* Updated anvil-daemon to periodically call tools/anvil-update-states. Also upadted anvil-daemon to delay daily jobs by 2 hours except for the dashboard with the highest sorted UUID to minimize dual runs of tasks that only need to run once per day per cluster.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-04 00:02:19 -05:00
Digimer
e8d15112da * Fixed a bug in anvil.js where the state of a link always said 'up', even when it was down.
* Fixed a couple logging bugs in System->call().
* Fixed a bug in anvil-daemon where it was trying to setup setuid-C wrappers on non-dashboards.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-03 01:23:31 -05:00
Digimer
90f5bf49d5 * Updated Network->load_interfces() to only assign a 'changed_order' to real interfaces.
* Fixed a bug in System->generate_state_json() where interfaces connected to a bridge were constantly having their 'network_interface_bond_uuid' cleared and reset.
* Finished (for now) the jquery code to update the network interface list when preparing the network interface configuration of a new node or DR host.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-12-03 00:30:26 -05:00
Digimer
0dfd2ddbf0 * Renamed 'ip' to 'ip_address' in Striker->parse_all_status_json() and related functions.
* Got the menu for mapping a host's network displaying (much work still to be done).
* Updated the anvil.js funtion to run dependent on the page being shown. For the main menu, the json is now properly reread and display updated as json content changes.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-29 23:37:11 -05:00
Digimer
0b2439edc3 * Finished the jquery/css stuff for showing the list of hosts that can have their network configured.
* Fixed unclear error logging in Network->find_matches().
* Updated System->generate_state_json() Striker->parse_all_status_json() to determine if another machine can be reached from the local dashboard. If it can be reached, the first matching interface and IP are recorded.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-27 23:26:21 -05:00
Digimer
c608ebb232 * Added bonds -> bond_bridge_uuid to track which bridge, if any, that a bond is connected to. Updated Network->load_interfces() to record this.
* Added missing foreign key references to the SQL schema.
* Added support to tools/anvil-update-states to connect bonds to bridges, as appropriate.
* Finished the logic in test.pl to pull the network data (with connections between bridges, bonds and interfaces) needed for the WebUI.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-26 00:15:37 -05:00
Digimer
0cf13b6387 * Worked out more of the logic in test.pl for displaying the JSON data.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-22 18:14:25 -05:00
Digimer
82bc25c78d * Adding the test.pl script. Even though it doesn't relate to the program, it's used to sort out logic and the code should be committed, until mainline development is done.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 19:47:21 -05:00
Digimer
7aaef3d6d5 * Fixed bugs in Striker->parse_all_status_json() and System->generate_state_json() where the connected interfaces to bridges and bonds wasn't be stored properly.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 19:23:16 -05:00
Digimer
aba79cc457 * Added a check in Striker->parse_all_status_json() to set the connected interface types.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 14:07:03 -05:00
Digimer
54de5466cf * Renamed bond_active_slave to bond_active_interface.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-21 10:55:21 -05:00
Digimer
12b0cc28f0 * Renamed 'primary_slave' to 'primary_interface' for the bond table.
* Got more work done on parsing the all_status.json (and fixing related bugs).

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-20 01:45:16 -05:00
Digimer
d5672e77eb * Standardized 'subnet_mask' over 'subnet' naming throughout.
* Created (and haven't finished yet) Striker->parse_all_status_json().

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-19 02:13:19 -05:00
Digimer
b2ff4c4c53 * Created Network->get_company_from_mac() that converts a MAC address to the company name that owns it.
* Added recording the last-change order for network interfaces in System->generate_state_json() so that the most recently unplugged and plugged back in interfaces can be tracked.
* Worked out a faster way to ping scan subnets with nmap in striker-scan-network. Dropped average scan time from 35 minutes to 4~5 minutes for a /16 subnet.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-15 18:29:07 -05:00
Digimer
e3a8c1a01d * Created System->generate_state_json() that reads, parse and writes out the network status of all known machines on a given Striker database.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-14 23:58:39 -05:00
Digimer
387c03aa7d * Added more text for the pending JSON state file generation.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-13 21:29:51 -05:00
Digimer
7cf1db3b37 * Added strings for making size suffixes translatable and updated Convert->bytes_to_human_readable to use them.
* Fixed a bug in tools/anvil-update-states where the MAC of an interface that is the backup in an active-backup bond would be the MAC of the active member instead of its real MAC.
* Fixed a bug in Convert->add_commas() where a passed in value of '0' returned an empty string.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-12 16:27:01 -05:00
Digimer
b9e8d2a971 * Added suffixes for data sizes.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-11 18:18:29 -05:00
Digimer
d0006cc4d3 * Changed a bunch more variables to standardize on 'mac_address' instead of 'mac'.
* Added ' --best --allowerasing' to the dnf call in tools/anvil-update-system (found it was needed when 'yum-utils' replaced 'dnf-utils' in RHEL 8.1 release).

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-11 11:17:02 -05:00
Digimer
70776964d6 * Added packages needed to support offline machines to striker-manage-install-target.
* Cleaned up the returned 'return_code' in System->call() when background is true.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-08 14:49:49 -05:00
Digimer
ea27f88bcc * Removed the 'bridge_interfaces' table and related database methods. This was found to be duplicate, given that 'network_interfaces' has references to parent bonds and bridges.
* Renamed 'bridges' -> 'bridge_mac' to 'bridge_mac_address' to be consstent with other MAC address column names.
* Finished Network->load_interfces().
* Updated anvil-update-states to check for interfaces under bridges that are missing their 'network_interface_bridge_uuid' reference UUID.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-07 23:40:02 -05:00
Digimer
628f7faa45 * Updated the RPM spec file to generate '/etc/anvil/type.X' files to directly indicate the machine type. Updated System->get_host_type() to check for these files directly, falling back to parsing the host name if they don't exist.
* Created Database->get_hosts_info() (though it's not at all finished) that will write out a unified JSON file contain all data known about all hosts/Anvil! systems. This will be later used to create the WebUI parts.
* Also created, but also not finished, Network->load_interfces() that will work sort of like ->load_ups, but include all interfaces regardless of if they have an IP or not.
* Fixed a bug where the new bridge_interface_note parameter didn't exist in the Database->insert_or_update_bridge_interfaces() method.
* Updated anvil-update-states() to only write out the JSON/XML files if it's running on a dashboard. For nodes and DR hosts, it just needs to update the database.
* Created a new hook in anvil-daemon that will call tasks on a machine that is configured.
* As per RHEL 8.1 release notes, changed the package 'dnf-utils' to 'yum-utils' in the packages to load for install target repos.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-06 11:26:21 -05:00
Digimer
077977ad9c * Fixed tools/anvil-update-states so that it properly removed ip_addresses no longer assigned to a host. Also merged 'network::interfaces::by_name::${interface}' with 'network::local::interface::${interface}' for storing discovered interfaces.'
* Added 'ip_address_note' to the 'ip_addresses' table as there was no column convenient for flagging as DELETEd.
* Added 'uuid' to Database->insert_or_update_file_locations() and ->insert_or_update_files(), and actually used it in all ->inser_or_update_X() methods.
* Added 'delete' as a parameter to Database->insert_or_update_ip_addresses() to allow simple deletion of a referenced IP address.
* Addressed a few 'undefined variable' errors.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-11-02 00:02:36 -04:00
Digimer
517771594e * Updated to release -28 RPM.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-28 11:04:57 -04:00
Digimer
197f5ed90e * Finished the automatic archving of the database! New in M3; Archving happens on dashboards only, and whichever database triggers an archive will archive all data in any connected databases. This is a departure from where each host archived it's own data, but will avoid putting a disk or CPU load on server hosts.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-28 02:27:26 -04:00
Digimer
650dc6eb80 * Finished Storage->compress(). Also added some missing 'port' parameters (reads and passes) in a few different Storage methods.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-27 17:19:03 -04:00
Digimer
deb8965a38 Merge branch 'master' of github.com:digimer/anvil
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 20:43:17 -04:00
Digimer
08176a5711 * Updated Database->_archive_table() to remove double-// in paths.
* Started work on Database->compress.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 20:10:02 -04:00
Digimer
b29bfd4c59 * Added some logging to the archive method.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 20:07:03 -04:00
Digimer
b436500a54 * Started working on getting database archiving working. It's close to done, but needs testing/debugging and other finery.
* Created Database->get_host_from_uuid() that takes a host UUID and returns the host's name.
* Reworked Network->find_matches() to return both the match's IP and subnet.
* Finished getting tools/striker-initialize-host to add all known peers to the target, using IPs on the target's available subnet.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-26 01:53:52 -04:00
Digimer
b1a175baa0 * Finished Network->load_ips().
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-23 11:10:07 -04:00
Digimer
934c9b1286 * Updated logging to now log anything with 'priority' set to a new 'anvil.alert.log' file (while still also logging as normal to anvil.log). This should make it easier to watch for alert messages.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-22 21:17:30 -04:00
Digimer
f750e087f6 * Prepped .spec for -27 release.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-20 23:09:17 -04:00
Digimer
af6e2c076d * Fixed a tricky deep recursion bug in Network->is_local when the passed in host was an empty string. Also created a cache system where a host name that has been checked before is immediately returned, without needing to run through the logic in 'is_local', which gets called quite frequently.
* Updated the loop detection logic in Log->entry where processing large strings was triggering it when it shouldn't.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-20 22:55:49 -04:00
Digimer
32bcdbe6d3 * Removed Network->is_remote, standardized on Network->is_local, and flipped calls to it to be more sensible (is_local -> local call -> else remote call). Also fixed a deep recursion issue with ->is_local where, given that it logs (which calls Storage methods which have local/remote invocations), would loop.
* Fixed a bug where '$target' being preset to 'local' was causing bad calls to 'Remote->call'.
* Updated Storage->change_mode and -> change_owner to work locally and on remote hosts.
* Barely started work on striker->process_anvil_menu().

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-19 00:57:33 -04:00
Digimer
a7f93c59ea * Reworked striker-parse-oui and striker-scan-network to always lower-case the MAC address.
* Updated striker-scan-network to only run once per day unless --force or a given --network is used. This avoids repeated scans when the anvil-daemon restarts frequently for whatever reason.
* Fixed (for real this time) Convert->time's handling of the 'long' parameter.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-18 11:40:38 -04:00
Digimer
4d0a02ce74 * Fixed a bug in Database->get_local_uuid() where ->is_local() was being called incorrectly.
* Added job parsing to tools/striker-parse-oui and tools/striker-scan-network, and enabled them in anvil-daemon.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-17 13:08:11 -04:00
Digimer
1d13e669a7 * Created the new tools/striker-scan-network tool that ping scans a network range and records the discovered hosts in the new 'mac_to_ip' table. Also created Database->insert_or_update_mac_to_ip() to handle the new table.
Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-16 23:23:02 -04:00
Digimer
7e960f1632 * Created the 'oui' database table that stores the parsed OUI data, as processed by the new 'tools/striker-parse-oui' tool. Also created Database->insert_or_update_oui() to handle inserts and updates.
* Fixed a bug in Convert->time() where the suffix was long when it should have been short, and vice-versa.
* Updated Network->download() to check if the target file exists and, if so, to abort unless 'overwrite' is given or the existing file is 0-bytes long. Also updated it to not exit on immediate error after the wget call and instead check to see if a zero-byte file exists and remove it, if so.
* Created Validate->is_hex() to check hexadecimal strings.
* Updated Words->clean_spaces() to remove MS-DOS-style ^M cr/lf characters.
* Updated anvil-daemon to have a section for periodic tasks that run daily, and added striker-parse-oui as well as moved striker-manage-install-target refresh to that check. Also made those tools run on dashboards only.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-16 01:57:59 -04:00
Digimer
7cdd2f60e9 * Created Network->download() to handle downloading a file on the local system. Created ->bridge_info() to parse 'bridge' output. Created ->load_ips() to load IP address information from the database (as opposed to ->get_ips() which queries a system).
* Created Database->insert_or_update_bridge_interfaces() to handle updating the new bridge_interfaces database table that records which network interfaces are connected to which bridges.
* Added 'bridge_mac' and 'bridge_mtu' to the 'bridges' table.
* Started Server->map_network which will, eventually, try to map MAC addresses to IPs and record server -> vnetX -> bridge data. Made getting a server status target-dependent.
* Worked on anvil-update-states to parse and record bridge data.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-12 00:32:32 -04:00
Digimer
183d2d9cce * Updated Storage->backup to append a short UUID to the timestamp to prevent issues if the same file is backed up twice in the same clock second. Also fixed a bug with remote_user parameter not having a default.
* Finished the detection of and handling of initialization of a host when the host has no Internet access.
* Disabled (for now) anvil-daemon's check_ssh_keys function.
* Fixed a couple small bugs elsewhere.

Signed-off-by: Digimer <digimer@alteeve.ca>
2019-10-04 22:36:39 -04:00