Hi, I read this thread a few minutes ago. I really like the short address http://trinitydesktop.mirrorservice.org/trinity/ - if such an address can be considered stable (it was mentioned that it is testing), I would prefer to use this URL. We currently have the mirror list configuration as a simple configuration file, so it's easy to change - see: https://mirror.git.trinitydesktop.org/gitea/TDE/tdemainweb-core/src/branch/master/mirrors.json A small disadvantage compared to the current state will be that when using one record for your mirror system, the redirector may obtain inaccurate information about the state of the mirror, because it will not be certain which of your backends has been contacted. Specifically, the redirector may get a different state than the real user will subsequently receive. Our redirector not only checks the presence of files on the selected mirror, but also checks the up-to-dateness of files before redirecting the user. When there are different states for the redirector and for the real user, there may be a redirector performing the redirection and the user receiving a code 404 ... which is exactly the state our redirector is trying to prevent. Checking the up-to-dateness of files is useful, for example, for APT packaging files. However, this should not be a major problem under normal circumstances. Cheers -- Slávek On Sunday 19 of July 2020 20:22:37 Mike Bird wrote: > Hi Slávek, > cc: Tim Bishop > > Very helpful information here from Tim. > > In short, could we change the mirror list on > https://www.trinitydesktop.org/mirrorstatus.php > to replace his copernicus and kuiper with just > http://www.mirrorservice.org/sites/trinitydesktop.org/trinity/ > while our redirector could continue to use copernicus and kuiper? > > Primary mirror would not be affected by this change. > > Thanks, > > --Mike > > On Sun July 19 2020 03:25:35 Tim Bishop wrote: > > Hi Mike, > > > > The two servers sync independently. Whilst this is not an ideal > > situation, and can lead to them having slightly out of step content, > > it's the simplest approach and requires the least development and > > maintenance work. In theory they should sync within an hour of each > > other, so the likely reason for them to have different files for an > > extended period is that one of them is failing to sync. Obviously this > > is something that's quite easy to investigate at the time, but > > impossible to work out later. So if you see this please let us know at > > the time so we can see why it's happening. > > > > The load balancer doesn't take much action regarding this. It doesn't > > know which mirrors are up to date or not, just whether they're > > functioning. However, it does have "stickiness" enabled, which means a > > client from the same IP should repeatedly hit the same backend unless > > there's a failure. So they should at least get consistent content. > > > > My main gripe is about publishing the details of the backend systems > > directly. This leads to people using them not via the load balancer > > and then complaining when one is down. In your case you have a smart > > redirector in front, so presumably that'd deal with that problem > > without inconveniencing the users. I don't have any problem with you > > doing that but I'd be happier if the backend URLs weren't so obviously > > advertised. > > > > The period of downtime in early June was because we had a complete > > data centre outage for the first time in over 10 years. The UPS > > systems were being upgraded and a failure occurred which powered > > everything off - the systems supposed to save us were the ones that > > failed us :-( The Mirror Service sadly isn't a priority system for the > > University, compared to core teaching and research systems, so we had > > to wait until full power was restored before we could turn it back on. > > > > That issue also killed copernicus, or more specifically its RAID > > controller. I have now replaced that and spent the past couple of > > weeks rebuilding it and resyncing the data. This has proved quite time > > consuming but it is starting to come back online this weekend. > > > > Tim. > > > > On Sat, Jul 18, 2020 at 11:35:18AM -0700, Mike Bird wrote: > > > Hi Tim, > > > > > > Thank you for mirroring TDE! > > > > > > I don't maintain that web page so this is not a direct response > > > to your request but rather a request for further information. > > > > > > FYI https://www.trinitydesktop.org/mirrorstatus.php is not the > > > normal route whereby users retrieve files. Users work instead > > > via a smart redirector http://mirror.ppa.trinitydesktop.org/ > > > > > > The status page is more of a debugging tool. > > > > > > We monitor mirrors to determine when they have pulled updates. > > > There have been several extended periods when copernicus and > > > kuiper have held different files. We use this information in > > > the smart redirector to only send user traffic to mirrors which > > > are known to have the file the user needs. > > > > > > I was wondering to what extent the UK Mirror Service load balancer > > > addresses this as the last time I saw copernicus rsync from us > > > was June 19th? And earlier in June there was a period of a few days > > > when neither kuiper nor copernicus were updating? > > > > > > Thanks again! > > > > > > --Mike