trinity-devel@lists.pearsoncomputing.net

Message: previous - next
Month: July 2018

Re: [trinity-devel] Re: Mirrors and PSB

From: Slávek Banko <slavek.banko@...>
Date: Sun, 29 Jul 2018 11:09:22 +0200
On Friday 27 of July 2018 15:38:59 Michael Meier wrote:
> (sorry if I broke the threading, not having a mail to reply to here)
>
> > The primary mirror has roughly 240GB of which 192GB is currently in 
use.
> >     This can be increased but of course none of us want to spend more
> > money on renting data center disk space than necessary.  I don't know 
how
> > much disk space is available for TDE on the secondary mirrors.
>
> The question is, aren't most of these 192 GB useless by now?
> Looking at what we currently have in our mirror dir, it seems there is a
> lot of stuff there that could be gotten rid of. To me it really doesn't
> make much sense to keep e.g. the 3.5.12-release from 2010 available on
> all mirrors. Or a Maverick-iso from 2010. Other projects, e.g. CentOS,
> solve this by only keeping the currently maintained versions on the main
> mirror network, and at some point moving the things that are no longer
> maintained to an archive site. This has a lot of advantages: That
> content usually gets _extremely_ few requests, so it uses up far more
> traffic to keep the mirrors updated than the mirrors will ever receive
> requests for it. It also reduces the space usage on the mirrors. It
> saves everyone space and traffic. And the few requests for archive
> content can be handled by a server with very low bandwidth.
>

Yes, some data are seldom used, some suitable for removal (for example, 
git-images, old development ISO files in trinity/cdimages/ubuntu).

To postpone old packages to the archive, I see two problems here:

1) Repositories, although for old versions, are connected to QuickBuild 
repositories. Here is a good reason to keep this state. Indeed, 
repository for v3.5.12 have a very reduced number of distributions that 
have been preserved.

2) An appropriate location for a possible archive would probably be the 
server on the primary site. But it would mean that all the downloads from 
the archive would lead to overloading the bandwidth to the primary 
server. But that's something we're trying to prevent - just by publishing 
content on mirrors. Only if the access speed to the archive was 
artificially reduced, but that does not seem like a good idea. Such an 
archive would be counterproductive.

> > I am not personally in direct contact with any of the other mirrors, 
even
> >     though they all pull from us.  Contact between the mirror admins 
has
> > AFAIK always been through Tim.
> > The primary mirror uses rsync rather than apt-mirror, and I suspect 
the
> >     same is true for the other mirrors.
>
> We're running one of these mirrors at ftp.fau.de. Both your assumptions
> are correct for us.
>
> > (From Sl�veks mail)
> > If anyone is interested in synchronizing Preliminary Stable Builds or
> > Preliminary Testing Builds, just let me know and I can also make them
> > accessible via rsync
>
> We would gladly mirror this _if_ there is a demand for it, i.e. we don't
> spend more traffic syncing the mirror than the mirror will ever see
> requests.
>
> > As a TDE user myself I would find it convenient but not critical if
> > there were fewer differences between PSB and Stable.  Ideally they
> > would be in the same repo pool - like Debian testing and stable.
>
> As a TDE user myself, I would very much support the idea of putting the
> PSBs, signed with a proper trinitydesktop.org and not a personal key,
> onto the official mirror network. Rename them into "testing" builds so
> it's clear what they are (that name is a lot more self-explaining than
> "PSB").
>

As I mentioned above, all official repositories (signed by the official 
Trinity key) are managed on QuickBuild. As I explained in previous 
emails, Preliminary Stable Builds are created independently of 
QuickBuild - independent builders (used pbuilder on my builders), 
independent repository maintenance (used reprepro on my server).

That's why PSB repositories simply can not be signed with an official 
Trinity key that is integrated with QuickBuild. I could create some 
general GPG key for signing repository instead of my personal, but it 
would still be a key other than the official Trinity key. So it seems 
like a futile change.

Regarding the propose to rename from 'stable' to 'testing' builds, this 
change is not possible. Existing Preliminary Stable Builds are built on 
a 'stable' branch (now r14.0.x) == this is preliminary packages for the 
next maintenance relase - therefore 'stable' in the repository name. 
Additionally, the second repository named Preliminary Testing Builds has 
already been prepared. This second repository is built on a 'master' 
branch == this is preliminary packages for the next major release, which 
rightfully deserves the naming of 'testing'. The official announcement of 
this 'testing' repository can be expected soon.

> Regards,

Thank you for good ideas. I hope your proposed clean up of old unnecessary 
things will be successful. Tim is the only one who can do this. I hope 
you understand why some of the proposals can not be done.

Thank you for your efforts in supporting the project.

Cheers
--
Sl�vek