* Got tools/striker-sync-shared to pick up 'upload::move_incoming' jobs, move the uploaded file to /mnt/shared/files/, copies it to peer dashboards, adds it to the 'files' table and adds it to 'file_locations'.
* Reworked the 'file_locations' table to now map files to Anvil! systems, not hosts. It simply tracks if a given file should be on Anvil! members or not. Later, striker-sync-shared on the Anvil! members will pull the file down.
* Updated Storage->get_file_stats() to record the file's mimetype.
* Fixed up a few issues in cgi-bin/upload.pl.
Signed-off-by: Digimer <digimer@alteeve.ca>
file_directorytextnotnull,-- This is the directory that the file is in.
file_sizenumericnotnull,-- This is the file's size in bytes. If it recorded as a quick way to determine if a file has changed on disk.
file_md5sumtextnotnull,-- This is the sum as calculated when the file is first uploaded. Once recorded, it can't change.
file_typetextnotnull,-- This is the file's type/purpose. The expected values are 'iso', 'rpm', 'script', 'disk-image', or 'other'.
file_typetextnotnull,-- This is the file's type/purpose. The expected values are 'iso', 'rpm', 'script', 'disk-image', or 'other'. If set to 'DELETED', the file will be removed from disk.
file_mtimenumericnotnull,-- If the same file exists on different machines and differ md5sums/sizes, the one with the most recent mtime will be used to update the others.
-- NOTE: When an entry is made here, the next time files are checked on a machine and an entry doesn't exist
-- on disk, the file fill be found (if possible) and copied to the houst. Only machines on the same
-- subnet are searched. Of course, if a URL is given (or a file is uploaded over a browser), the file
-- will be sourced accordingly. The search pattern is;
-- Nodes; 1. Check for the file on the peer.
-- 2. Check for the file on Strikers, in alphabetical order.
-- 3. Check for the file on DR host, if available.
-- 4. Check other nodes, in alphabetical order.
-- 5. Check other DR hosts, in alphabetical order.
-- Striker; 1. Check for the file on other Strikers, in alphabetical order.
-- 2. Check for the file on DR hosts, if available
-- 3. Check for the file on Anvil! nodes.
-- DR Host; 1. Check for the file on Strikers, in alphabetical order.
-- 2. Check for the file on Anvil! nodes.
-- * If a file can't be found, it will try again every so often until it is found.
-- * When a file is found, it is copied to '/mnt/shared/incoming'. Only when the file has arrived and
-- the md5sum matches. At this point, it is moved into the proper directory.
-- How new files are handled;
-- * When uploading a file from a Striker web interface, or when creating an ISO from physical media,
-- it will be dropped into /mnt/shared/incoming. Once there, the user will have the option of pushing
-- the file to an Anvil! system. ISOs and scripts will go to both nodes (and the DR host, when
-- needed).
-- * Repo RPMs are sync'ed to all peer'ed dashboards, but not sent to hosts (they are used during the
-- initial host setup).
-- * Special Note: Definition files are stored in the database and written out as needed to the
-- nodes/DR host.
--
-- This tracks which files should be on which machines.
-- This tracks which files on Strikers should be on Anvils!
CREATETABLEfile_locations(
file_location_uuiduuidnotnullprimarykey,
file_location_file_uuiduuidnotnull,-- This is file to be moved to (or restored to) this machine.
file_location_host_uuiduuidnotnull,-- This is the sum as calculated when the file_location is first uploaded. Once recorded, it can't change.
modified_datetimestampwithtimezonenotnull,
file_location_uuiduuidnotnullprimarykey,
file_location_file_uuiduuidnotnull,-- This is file to be moved to (or restored to) this machine.
file_location_anvil_uuiduuidnotnull,-- This is the sum as calculated when the file_location is first uploaded. Once recorded, it can't change.
file_location_activebooleannotnulldefaultTRUE,-- This is set to true when the file should be on Anvil! machines, triggering rsyncs when needed. When set to false, the file will be deleted from members, if they exist.
<keyname="error_0167">The resource: [#!variable!resource!#] in the config file: [#!variable!file!#] was found, and we were asked to replace the 'scan_drbd_resource_uuid' but the new UUID: [#!variable!uuid!#] is not a valud UUID.</key>
<keyname="error_0168">The 'fence_ipmilan' command: [#!variable!command!#] does not appear to be valid.</key>
<keyname="error_0169">The Anvil! UUID: [#!variable!anvil_uuid!#] doesn't appear to exist in the database.</key>
<keyname="error_0170">Unable to move an uploaded file from the: [#!data!path::directories::shared::incoming!#] directory as a file name wasn't set (or failed to parse) from the 'job_data' in the job: [#!variable!job_uuid!#].</key>
<keyname="error_0171">Unable to move the uploaded file: [#!variable!file!#], it doesn't appear to exist.</key>
<keyname="error_0172">Unable to move the uploaded file: [#!variable!file!#] to: [#!variable!target_directory!#]. The cause of the failure should be in the logs.</key>
<!-- Table headers -->
<keyname="header_0001">Current Network Interfaces and States</key>
@ -422,6 +425,8 @@ Failure! The return code: [#!variable!return_code!#] was received ('0' was expec
<keyname="job_0129">Completed joining the #!string!brand_0002!#.</key>
<keyname="job_0130">No job was found that needs to be run.</key>
<keyname="job_0131">Reconnecting will start a synchonization of the database. This step might take a while to complete, please be patient.</key>
<keyname="job_0132">Sync Uploaded File</key>
<keyname="job_0133">This moves an uploaded file from the: [#!data!path::directories::shared::incoming!#] directory to: [#!data!path::directories::shared::files!#] directory, adds it to the Anvil! database, and pushed it out to other systems.</key>
@ -1376,6 +1381,13 @@ About to try to download aproximately: [#!variable!packages!#] packages needed t
# scan_drbd_resource_uuid = #!variable!uuid!#
</key><!-- Translation note: The format of [# scan_drbd_resource_uuid = #!variable!uuid!#] is important, don't alter it. -->
<keyname="message_0190">Preparing to provision a new server.</key>
<keyname="message_0191">Processing an uploaded file.</key>
<keyname="message_0192">Moving the file: [#!variable!file!#] to: [#!data!path::directories::shared::files!#].</key>
<keyname="message_0193">Calculating the md5sum. This can take a little while for large files, please be patient.</key>
<keyname="message_0194">The md5sum is: [#!variable!md5sum!#]. Storing details in the database.</key>
<keyname="message_0195">Copying the file over to: [#!variable!host!#]. Please be patient, this could take a bit for large files.</key>
<keyname="message_0196">Registering the file to be downloaded to the Anvil!: [#!variable!anvil_name!#]. Anvil! members will sync this file shortly. Member machines that are not online will sync the file when they do return.</key>
<keyname="message_0197">Upload is complete!</key>
<!-- Success messages shown to the user -->
<keyname="ok_0001">Saved the mail server information successfully!</key>
Digimer
4 years ago