trinity-devel@lists.pearsoncomputing.net

Message: previous - next
Month: March 2020

Re: [trinity-devel] Embedded metadata × reproducible builds

From: Slávek Banko <slavek.banko@...>
Date: Fri, 13 Mar 2020 02:17:43 +0100
On Thursday 12 of March 2020 14:25:32 Sl�vek Banko wrote:
> On Thursday 12 of March 2020 03:22:10 Michele Calgaro via trinity-devel
>
> wrote:
> > On 2020/03/12 01:50 AM, Sl�vek Banko wrote:
> > > Hi all,
> > >
> > > as you probably know, if tdelibs are compiled with elf editor
> > > support (WITH_ELFICON), the application icon, TDE information and
> > > git repository information are embedded into libraries and binaries
> > > during CMake build.
> > >
> > > This data, on the one hand, provides user convenience - better to
> > > see the application icon than the general binaries icon. On the
> > > other hand, they provide information to developers - for example,
> > > when reporting creashes.
> > >
> > >
> > > It is possible that you have also heard about the activity
> > > Reproducible builds, which seems to us as a very good idea. See:
> > >
> > > https://reproducible-builds.org/
> > > https://wiki.debian.org/ReproducibleBuilds
> > >
> > >
> > > Currently, the metadata that are embedded has a Compilation
> > > Date/Time entry. This is set to the current date and time at the
> > > time of building the binary package. And this is a problem because
> > > it makes it impossible to achieve reproducible builds.
> > >
> > > My suggestion is that next to the ".tdescmmodule" and
> > > ".tdescmrevision" files we could have a ".tdescmdatetime" file
> > > containing the git commit date and/or a ".tdepackagedatetime" file
> > > containing the date the source package was created for distribution.
> > > For embedded metadata, this fixed time would be used instead of
> > > variable time.
> > >
> > > What is your opinion?
> > >
> > > Cheers
> >
> > Hi Slavek,
> > glad that you have brought up this issue, since it is something I have
> > experienced several times when checking the "drop automake" PR in the
> > last few months.
> > I also feel it is good to have reproducible builds. We already have
> > the ".tdescmmodule" and ".tdescmrevision" which contains the git
> > commit hash the package was build from. Nevertheless we are missing
> > the
> > tde-packaging hash in those files.
> > Rather than having too many .tdescmXXXXX files, I propose we use a
> > single .tdegitinfo file which includes module name, repo git hash and
> > packaging repo hash.
> > I don't see any need for any date in the package once we have the git
> > hash info.
> >
> > Cheers
> >   Michele
>
> Thank Michele, it is a good idea that instead of many files we can
> consolidate git repository information into one file. I will prepare
> patches / pull-requests in this sense.
>
> Since the tde-packaging repository information and package information
> for each distribution are independent, a separate package information
> file will be required.
>
> At the same time, git hash for tde-packaging is not enough for us,
> because there is also interesting information about the distribution. In
> addition, here can be forced rebuild without any changes in the git
> repositories - as an example here is the 'libr' with dependency on
> binutils. Therefore, it is still useful for us information about
> date+time and/or the version of the package for a specific distribution.
>
> Cheers

The puzzle pieces are as follows:

1. The create_tarball script instead of individual .tdescmmodule 
and .tdescmrevision files creates a single .tdescminfo file that contains 
the SCM module name, revision, and date of the last commit:

https://mirror.git.trinitydesktop.org/gitea/TDE/scripts/pulls/2


2. DEB packaging ensure that a .tdepkginfo file with package metadata is 
generated during build:

https://mirror.git.trinitydesktop.org/gitea/TDE/tde-packaging/pulls/63


3. The CMake macro retrieves information from .tdescminfo as well as 
information from .tdepkginfo in source or binary directory. And then use 
TDE_PKG_DATETIME, or TDE_SCM_MODULE_DATETIME, or the current date:

https://mirror.git.trinitydesktop.org/gitea/TDE/tde-common-cmake/pulls/32


As a result of these pull requests, for deb packages, the date for embedded 
metadata will consistently contain information about the time the source 
package was created. So it looks like a good starting point.

So far my initial tests are giving strange results. When I made two 
independent chroots on the builder (both bullseye@amd64) and trying to 
build abakus package manually using dpkg-buildpackage, I repeatedly get 
identical binary packages - the only difference is the "buildinfo" file 
that contains the build date. When I use automated building with sbuild, I 
get different binary packages. For now I don't know the cause.

Cheers
-- 
Sl�vek

Attachments: