On Monday 30 of July 2018 04:18:03 Sl�vek Banko wrote: > On Sunday 29 of July 2018 12:15:07 Mike Bird wrote: > > On Sun July 29 2018 02:09:22 Sl�vek Banko wrote: > > > As I mentioned above, all official repositories (signed by the > > > official Trinity key) are managed on QuickBuild. As I explained in > > > previous emails, Preliminary Stable Builds are created > > > independently of QuickBuild - independent builders (used pbuilder > > > on my builders), independent repository maintenance (used reprepro > > > on my server). > > > > > > That's why PSB repositories simply can not be signed with an > > > official Trinity key that is integrated with QuickBuild. I could > > > create some general GPG key for signing repository instead of my > > > personal, but it would still be a key other than the official > > > Trinity key. So it seems like a futile change. > > > > > > Regarding the propose to rename from 'stable' to 'testing' builds, > > > this change is not possible. Existing Preliminary Stable Builds are > > > built on a 'stable' branch (now r14.0.x) == this is preliminary > > > packages for the next maintenance relase - therefore 'stable' in > > > the repository name. Additionally, the second repository named > > > Preliminary Testing Builds has already been prepared. This second > > > repository is built on a 'master' branch == this is preliminary > > > packages for the next major release, which rightfully deserves the > > > naming of 'testing'. The official announcement of this 'testing' > > > repository can be expected soon. > > > > Hi Sl�vek, > > > > I am not yet understanding why we have so much duplication. Why is > > there not a "testing" release in the main repository instead of a > > having a main repo and a DEB PSB repo and a RPM PSB repo in different > > locations and using different tooling? (And soon in addition a new > > DEB testing repo?) > > > > I understand that PSB updates every two hours whereas the primary > > mirror currently checks for updates three hours after the completion > > of the previous run, but I could change the three hour delay to a ten > > minute delay without adversely impacting Tim's bandwidth. What is > > the average volume (GB) of DEB PSB updates published per day? > > > > The standard single Debian repository structure is well understood by > > all concerned. It reduces the number of downloads and upgrades > > needed for new minor releases and generally provides a smoother and > > more efficient transition from testing to release. Indeed Trinity > > successfully used a single repo incorporating trinity-nightly-builds > > for many years. > > > > --Mike > > Hi Mike, > > you mention "Indeed Trinity successfully used a single repo > incorporating trinity-nightly-builds for many years." But here is one > basic mistake. The repository trinity-nightly-builds is one of the > separate repositories on QuickBuild. At the time of publishing the new > release, the contents of this repository is on QuickBuild copied into > the official repository. And this is the moment when synchronization of > complete content to the mirror begins. There is no "It reduces the > number of downloads and upgrades needed for new minor releases and > generally provides a smoother and more efficient transition from > testing to release." It just does not work in that way. > > To make the situation even more complicated, you know that I also have > the repositories for preliminary stable packages that are on > QuickBuild. However, these repositories is not normally used by any > user just for reasons that were with the 'axis' repository. > > These repositories on QuickBuild I mainly use to prepare packages that > will then be published to the official repositories as a new release. > Into these repositories, I'm uploading identical source packages as I > uploaded into Preliminary Stable Builds repository. But on QuickBuild > packages must be built again because into QuickBuild is not possible to > upload binary packages, only source packages. > > It is just these repositories that contain the final packages for new > TDE release before they are published into the official repository. And > just these packages are downloaded into the cache on the redirector > already during the build == while the bandwidth to the primary server > is not loaded! And that is the main trick why this cache is so > important when releasing a new version. While to the primary mirror > synchronization starts at the moment of publishing packages, these > packages are already available on the redirector's cache before that > time. Simple and very effective. > > Since this synchronization of packages from the primary server to the > redirector cache is for deb packages, the advantate is to use > apt-mirror for this purpose. Apt-mirror works in a way that > repositories remain consistent during the update. This is why I use > apt-mirror for DEB packages, while for rpm packages I use rsync. > > Here is a small example of the trick. The current stable version is > R14.0.4. Therefore, listing the tdelibs folder from apt repository > displays packages for R14.0.4: > > http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/poo >l/main/t/tdelibs-trinity/ > > For example DSC file for Debian 8 (Jessie) is as follows: > > http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/poo >l/main/t/tdelibs-trinity/tdelibs-trinity_14.0.4-0debian8.0.0+0.dsc > > Since the builds of R14.0.5 final packages are currently underway and > these are continuously synchronized to the cache on the redirector, it > is now already possible to get files for the future R14.0.5 version. > For example DSC file for Debian 8 (Jessie) is as follows: > > http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/poo >l/main/t/tdelibs-trinity/tdelibs-trinity_14.0.5-0debian8.0.0+0.dsc > > Once the packages will be published to the official repository, the > folder listing starts showing new files, the meta-information > repositories will refer to new files, and these files will be ready in > the cache on the redirector. Simply because they are there in advance. > > Is it now clear what I mean? Is it now clear why the cache on the > redirector is very useful? And it is clear that keeping Preliminary > Stable Builds independent of QuickBuild is no duplication because the > same duplication is also going on at QuickBuild? > > Cheers Hi Mike, by the way, right now you can check the cache's advantage on the redirector. The packages were posted to the official repository a few hours ago. Mirror synchronization has not yet begun, but users can already install R14.0.5 from the official repository without causing load of bandwidth to the primary server. You will notice - the newly created folder, on the primary mirror folder does not yet exist: http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/pool/main/a/akode/ http://tde-mirror.yosemite.net/trinity/trinity-r14.0.0/debian/pool/main/a/akode/ ...but the redirector gives to the users HTTP code 200 == download file directly from the cache on the redirector: $ curl -I http://mirror.ppa.trinitydesktop.org/trinity/trinity-r14.0.0/debian/pool/main/a/akode/akode_14.0.5.orig.tar.xz HTTP/1.1 200 OK Date: Fri, 17 Aug 2018 07:18:42 GMT Server: Apache Content-Location: serve/trinity-r14.0.0/debian/pool/main/a/akode/akode_14.0.5.orig.tar.xz Last-Modified: Sat, 28 Jul 2018 07:46:50 GMT ETag: "5720a6e572e80" Content-Length: 166928 That is why I said that new versions are immediately available to users - thanks to cache == there is no reason to postpone official announcements for a long time. How do you like it? Cheers :) -- Sl�vek