Debian Bug report logs - #1089197
dh-r: Recommends/Suggests missing from rebuilt R packages

Reply or subscribe to this bug.

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Gioele Barabucci <gioele@svario.it>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: debrebuild: Recommends/Suggests missing from rebuilt R packages

Date: Sat, 7 Dec 2024 00:20:11 +0100

Package: devscripts
Version: 2.24.5
User: devscripts@packages.debian.org
Usertags: debrebuild
User: reproducible-builds@lists.alioth.debian.org
Usertags: toolchain
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Dear debrebuild developers,

many of the R packages that failed to reproduce on reproduce.d.n have 
lost their Recommends or Suggests fields during the rebuilding process.

```
 Installed-Size: 2837
 Depends: r-api-4.0, r-api-bioc-3.19, r-cran-ggplot2, r-cran-reshape2
-Suggests: r-cran-knitr
 Section: gnu-r
 Priority: optional
```

Example logs:

* <https://reproduce.debian.net/api/v0/builds/110379/diffoscope>
* <https://reproduce.debian.net/api/v0/builds/110412/diffoscope>

Regards,

-- 
Gioele Barabucci

Message #10 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Gioele Barabucci <gioele@svario.it>, 1089197@bugs.debian.org, Debian R Packages Maintainers <r-pkg-team@alioth-lists.debian.net>

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#1089197: debrebuild: Recommends/Suggests missing from rebuilt R packages

Date: Sat, 7 Dec 2024 10:54:32 +0000

[Message part 1 (text/plain, inline)]

control: reassign -1 dh-r
thanks

Hi Debian R Packages Maintainers,

reassigning this bug to dh-r, this is the full bug log so far:

On Sat, Dec 07, 2024 at 12:20:11AM +0100, Gioele Barabucci wrote:
> Package: devscripts
> Version: 2.24.5
> User: devscripts@packages.debian.org
> Usertags: debrebuild
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: toolchain
> X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org
> 
> Dear debrebuild developers,
> 
> many of the R packages that failed to reproduce on reproduce.d.n have lost
> their Recommends or Suggests fields during the rebuilding process.
> 
> ```
>  Installed-Size: 2837
>  Depends: r-api-4.0, r-api-bioc-3.19, r-cran-ggplot2, r-cran-reshape2
> -Suggests: r-cran-knitr
>  Section: gnu-r
>  Priority: optional
> ```
> 
> Example logs:
> 
> * <https://reproduce.debian.net/api/v0/builds/110379/diffoscope>
> * <https://reproduce.debian.net/api/v0/builds/110412/diffoscope>
> 
> Regards,
> 
> -- 
> Gioele Barabucci
> 

-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

The empty vessel makes the greatest sound. (William Shakespeare)

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Philip Rinn <rinni@inventati.org>

To: 1089197@bugs.debian.org

Subject: Re: debrebuild: Recommends/Suggests missing from rebuilt R packages

Date: Sat, 7 Dec 2024 13:25:46 +0100

[Message part 1 (text/plain, inline)]

Hi,

it seems dh-r is missing a build dependency on apt (which is neither essential nor build-essential, as I learned today ;-)).

Looking at the log https://reproduce.debian.net/api/v0/builds/110379/log the root cause seems

/bin/sh: 1: apt-cache: not found
grep-aptavail: command (apt-cache dumpavail) failed (exit status 127)
W: Ignoring specified R dependency: R (>= 2.10)
W: Cannot find a debian package for Suggests:knitr

which is a result of https://salsa.debian.org/r-pkg-team/dh-r/-/blob/master/dh/R.pm?ref_type=heads#L16

Best
Philip

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #24 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Philip Rinn <rinni@inventati.org>

To: 1089197@bugs.debian.org

Subject: Re: debrebuild: Recommends/Suggests missing from rebuilt R packages

Date: Sat, 7 Dec 2024 14:21:30 +0100

[Message part 1 (text/plain, inline)]

Hi,

unfortunately, it seems to be a little more complicated than just adding apt as build dependency. See the IRC log from #debian-reproducible

13:26 <jochensp> rinni: huh, adding apt does not seem to be enough, investigating
13:28 <rinni> jochensp: Oh, OK, I'll wait ... one probably also needs something apt-cache can look at ;-)
13:28 <jochensp> heh, that's probably it
13:28 <jochensp> but that is not really reproducible
13:29 <jochensp> as in, we would need to reproduce the apt repo stated from the time the package was compiled, I guess dh-r needs to find a different way there
13:33 <jochensp> well, we could add a snapshot.d.o URL based on the date in the build info but not sure that's a nice way
13:36 <rinni> Yeah, seems difficult (and my Perl skills a close to zero) ... dh-r needs to map CRAN package names to Debian package names somehow and drops suggested packages not in Debian. Not sure how to do that without knowing what's in the apt repo at the given time :-/
13:36 <jochensp> dh-python carries a list for that
13:37 <rinni> huh, how often is dh-python updated, with every new python package?
13:38 <jochensp> you can add overrides in a package in case it is outdated
13:38 <jochensp> not saying it is the best way just one example I know
13:39 <rinni> ah, yes, I remember now
13:39 <jochensp> https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/pydist/cpython3_fallback?ref_type=heads
13:40 <jochensp> there is no need for the suggested packages to be in Debian, btw
13:40 <jochensp> so maybe just add all of them and done
13:43 <rinni> true, but not nice and not so easy as there is not only CRAN but also BIOC and 'other' as possible package sources which results in different Debian package names r-cran- vs. r-bioc-
13:45 <jochensp> huh, how is the list used in the R world? Simply check all providers?
13:45 <rinni> see https://salsa.debian.org/rinni/dh-r/-/blob/master/dh/R.pm?ref_type=heads#L79-L113
13:45 <rinni> a package is either CRAN, BIOC or other
13:46 <jochensp> .oO(maybe that part should be dropped from the package name)

Best,
Philip

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #31 received at 1089197-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 1089197-close@bugs.debian.org

Subject: Bug#1089197: fixed in dh-r 20241218

Date: Wed, 18 Dec 2024 18:50:01 +0000

[Message part 1 (text/plain, inline)]

Source: dh-r
Source-Version: 20241218
Done: Andreas Tille <tille@debian.org>

We believe that the bug you reported is fixed in the latest version of
dh-r, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1089197@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Andreas Tille <tille@debian.org> (supplier of updated dh-r package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Wed, 18 Dec 2024 18:56:25 +0100
Source: dh-r
Architecture: source
Version: 20241218
Distribution: unstable
Urgency: medium
Maintainer: Debian R Packages Maintainers <r-pkg-team@alioth-lists.debian.net>
Changed-By: Andreas Tille <tille@debian.org>
Closes: 1089197
Changes:
 dh-r (20241218) unstable; urgency=medium
 .
   [ Andreas Tille ]
   * Keep r-base-dev if specified in d/t/control
   * Do not ignore *.r (lower case 'r') files in tests
   * Add architecture-is-64-bit, architecture-is-little-endian to restrict
     binary packages to 64bit and little endian
   * Replace apt-cache by rmadison
     Closes: #1089197
   * Standards-Version: 4.7.0
 .
   [ Alexandru Mahmoud ]
   * Update repo detection for new Bioconductor Repository field
Checksums-Sha1:
 dd9f7ce3ac96294a84a46d9cb3ebc26aaaa09a65 1738 dh-r_20241218.dsc
 a71332abe753bd715a0a7e4a0405c42281cd21de 45036 dh-r_20241218.tar.xz
 78435c831f8163f0693baffca44d221dd626843d 6261 dh-r_20241218_amd64.buildinfo
Checksums-Sha256:
 90a6f11833cecc0302e405fc7b89fd7225c8067f7a79604d85c9982546559218 1738 dh-r_20241218.dsc
 2a9ccdf3aeb7016307b292288221a3a991f769d57f6dff1d923f4d023da0f373 45036 dh-r_20241218.tar.xz
 4d050b8b885a28a2d57a7fc6ac9aec1c715706f42fe58ab7a7f3fe1bbafa808c 6261 dh-r_20241218_amd64.buildinfo
Files:
 342a971c4c8a414724228db558823276 1738 science optional dh-r_20241218.dsc
 e9eeac34a6b1bcc195ca76b98c212432 45036 science optional dh-r_20241218.tar.xz
 cb5a5653488b627053e56b78459ea565 6261 science optional dh-r_20241218_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJFBAEBCAAvFiEE8fAHMgoDVUHwpmPKV4oElNHGRtEFAmdjFpgRHHRpbGxlQGRl
Ymlhbi5vcmcACgkQV4oElNHGRtEb4A/9F4qsGA3Hgr/cTp+mvBfW+306MG7uDs+b
KW21GZXHRiBFjkjOIHgFKw6wRBLjBY0JWi2BCZj2lD20MRtYtkspT+ugqgHbo3An
QEobNe1GxZ6hnl0iUq0EG6X1aUClAGHOGQ6DJ21XGKsFi/hRhqcAU2JuwaIJ3opb
k+GdQqCzMAe8IWVQIzZEJohH+ZtZ2CT6RPLkQPUROYN23XFQ/JjY1GTQb51VSeJZ
DBapgKTcn3BRswcExEIPi4pB+14+bTcJEQYj6/56IylQXBDB8C8Z9I1ejZQvB2Z0
svVF48g2teBfiC4WluGbgyktVROP44/22IP/61S5FNrddFRlbmrIkG1oBfIEdYeX
+oYctPVW61TYBXN7Q6wJrNCdIpLkRZpjw3nXypByznsvzP3N+kFH7DGCStUa2Xnq
xTtIux6/3Zu6ND7wzzxr7NCpj24a3NJPw5Yx0pbzw6BwzJ2EEgIHAbF1k/bLFFYB
dt3vAUoNUJbdOawt5FoW3pb5X1KqlQV/nbPPzEHzEdee8X+b3Cm2UuONYDQsUDwK
pvasnzrrSRXBkNOrbBZQAmPhNjb1xbXrPnVuonoPwKncLNP8dby519wJR9og7CkB
9OlRAKdryZbM4ctJbeGSdbTek1SsTx/y5FtfO6oNwprh4iNhHrlMCAIbiQpT7orZ
jR493HDjxP0=
=3ys+
-----END PGP SIGNATURE-----

[Message part 2 (application/pgp-signature, inline)]

Message #44 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Philip Rinn <rinni@inventati.org>

To: 1089197@bugs.debian.org

Subject: Re: dh-r: Recommends/Suggests missing from rebuilt R packages

Date: Wed, 18 Dec 2024 21:16:21 +0100

[Message part 1 (text/plain, inline)]

Hi Andreas,

your upload doesn't fix the described issue.
The problem is that check_real_version_of_package() in dh/R.pm[1] uses 'grep-aptavail'. As parse_depends() calls check_real_version_of_package() to build Depends/Suggests/Recommend they depends on the status of the APT cache.

Hope that helps for context
Philip


[1]https://salsa.debian.org/r-pkg-team/dh-r/-/blob/master/dh/R.pm?ref_type=heads#L147-L170

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #49 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Philip Rinn <rinni@inventati.org>

To: debian-r@lists.debian.org

Cc: 1089197@bugs.debian.org

Subject: dh-r: Stop generating Depends/Suggests/Recommends from the apt cache

Date: Thu, 19 Dec 2024 11:31:14 +0100

[Message part 1 (text/plain, inline)]

[please keep me and the bug in cc]

Hi,

during the effort to reproduce Debian binary packages distributed via 
deb.debian.org (see https://reproduce.debian.net) it turned out that 
many/most/all R package couldn't be reproduced as some 
Depends/Suggests/Recommends where missing.

After digging it bit - the missing build-dependency on apt was 
unfortunately just a red herring -  it turns out, the problem is that 
check_real_version_of_package() in dh/R.pm[1] uses 'grep-aptavail' and 
thus the result depends on the state of the apt cache.
As parse_depends() (in dh/R.pm) calls check_real_version_of_package() to 
build Depends/Suggests/Recommend they depend on the status of the apt 
cache as well.

While this seems to be a quite elegant solution to the problem of 
mapping R package names to Debian package names, it turns out to be 
quite hard to reproduce the state of the apt cache for a given point in 
time.

Jochen reminded me of the solution dh-python chose [2] for that problem 
and I do think a (simplified) version of that would work for R packages 
as well.

The main idea is to have a mapping table embedded in dh-r which can be 
overrode on package level.

mapping table would basically be

<R package name> <corresponding Debian package name>
knitr r-cran-knir
ggplot2 r-cran-ggplot2
reshape2 r-cran-reshape2
...

containing a mapping for all packages in Debian unstable at the time of 
the dh-r upload.

This would of course mean that we'd need to keep that list moderately 
updated in the dh-r package.

As we can't guarantee that, we need the possibility to place a file in 
debian/ to amend/override the mapping table dh-r provides. (We could of 
course have a helper script creates that file for us.)


I'm quite sure, I miss some details here and I do entirely miss the 
context of why the current approach was chosen, but I hope, we can start 
a discussion on how to make it possible to reproduce R packages going 
forward.

Best
Philip


[1] 
https://salsa.debian.org/r-pkg-team/dh-r/-/blob/5d6dc04ba4f1f1b5db4bfaecc0e00893e46e5c72/dh/R.pm#L147-L170

[2] 
https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/pydist/README.PyDist

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #54 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Andreas Tille <tille@debian.org>

To: Philip Rinn <rinni@inventati.org>

Cc: debian-r@lists.debian.org, 1089197@bugs.debian.org

Subject: Re: dh-r: Stop generating Depends/Suggests/Recommends from the apt cache

Date: Thu, 19 Dec 2024 11:49:43 +0100

Hi Philip,

thank you for the detailed information.  I simply replaced all
occurences of apt-cache (which admittedly does *not* affect the creation
of a resulting binary package).

Your hint to[1] in your re-opening mail was helpful.

Am Thu, Dec 19, 2024 at 11:31:14AM +0100 schrieb Philip Rinn:
> [please keep me and the bug in cc]
> 
> Hi,
> 
> during the effort to reproduce Debian binary packages distributed via
> deb.debian.org (see https://reproduce.debian.net) it turned out that
> many/most/all R package couldn't be reproduced as some
> Depends/Suggests/Recommends where missing.
> 
> After digging it bit - the missing build-dependency on apt was unfortunately
> just a red herring -  it turns out, the problem is that
> check_real_version_of_package() in dh/R.pm[1] uses 'grep-aptavail' and thus
> the result depends on the state of the apt cache.
> As parse_depends() (in dh/R.pm) calls check_real_version_of_package() to
> build Depends/Suggests/Recommend they depend on the status of the apt cache
> as well.

That's correct.
 
> While this seems to be a quite elegant solution to the problem of mapping R
> package names to Debian package names, it turns out to be quite hard to
> reproduce the state of the apt cache for a given point in time.
> 
> Jochen reminded me of the solution dh-python chose [2] for that problem and
> I do think a (simplified) version of that would work for R packages as well.
> 
> The main idea is to have a mapping table embedded in dh-r which can be
> overrode on package level.
> 
> mapping table would basically be
> 
> <R package name> <corresponding Debian package name>
> knitr r-cran-knir
> ggplot2 r-cran-ggplot2
> reshape2 r-cran-reshape2
> ...
> 
> containing a mapping for all packages in Debian unstable at the time of the
> dh-r upload.
> 
> This would of course mean that we'd need to keep that list moderately
> updated in the dh-r package.

Well, in principle the package *names* can be calculated more or less -
we only need to know whether these are CRAN or BioConductor (or other)
packages.  However, maintaining dh-r after de facto every new r-*
package is a burden I would love to avoid.
 
> As we can't guarantee that, we need the possibility to place a file in
> debian/ to amend/override the mapping table dh-r provides. (We could of
> course have a helper script creates that file for us.)
> 
> 
> I'm quite sure, I miss some details here and I do entirely miss the context
> of why the current approach was chosen, but I hope, we can start a
> discussion on how to make it possible to reproduce R packages going forward.

Well, the package names in principle are taken from the metadata contained
inside the DESCRIPTION file every R package (they use the same name for
their source - so please do not mix up with Debian package).  There is
also the concept of versioned dependencies.  We simply check whether this
package / version is available in Debian.

I think what would be helpful for me to understand if you could give me
some example where the same build results in different packages.  I need
to admit that I might fail to understand the real consequences of the
code and an example would be pretty helpful, thought.

Kind regards
    Andreas.

> [1] https://salsa.debian.org/r-pkg-team/dh-r/-/blob/5d6dc04ba4f1f1b5db4bfaecc0e00893e46e5c72/dh/R.pm#L147-L170
> 
> [2] https://salsa.debian.org/python-team/tools/dh-python/-/blob/master/pydist/README.PyDist




-- 
https://fam-tille.de

Message #59 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Philip Rinn <rinni@inventati.org>

To: Andreas Tille <tille@debian.org>

Cc: debian-r@lists.debian.org, 1089197@bugs.debian.org

Subject: Re: dh-r: Stop generating Depends/Suggests/Recommends from the apt cache

Date: Thu, 19 Dec 2024 12:34:47 +0100

[Message part 1 (text/plain, inline)]

Hi Andreas,

On 19.12.24 at 11:49, Andreas Tille wrote:
>> The main idea is to have a mapping table embedded in dh-r which can be
>> overrode on package level.
>>
>> mapping table would basically be
>>
>> <R package name> <corresponding Debian package name>
>> knitr r-cran-knir
>> ggplot2 r-cran-ggplot2
>> reshape2 r-cran-reshape2
>> ...
>>
>> containing a mapping for all packages in Debian unstable at the time of the
>> dh-r upload.
>>
>> This would of course mean that we'd need to keep that list moderately
>> updated in the dh-r package.
> 
> Well, in principle the package *names* can be calculated more or less -
> we only need to know whether these are CRAN or BioConductor (or other)
> packages.  However, maintaining dh-r after de facto every new r-*
> package is a burden I would love to avoid.

The more I think about it, I guess we should actually _not_ have a list 
in dh-r but generate the Depends/Suggests/Recommends at the time of 
creating the _source_ package. So I'd suggest to have what I called 
'package level override' in my first mail (but filling d/control 
directly). [See the package example below for context of why.]

We'd probably need that list in dh-r for a migration period though.

>> As we can't guarantee that, we need the possibility to place a file in
>> debian/ to amend/override the mapping table dh-r provides. (We could of
>> course have a helper script creates that file for us.)
>>
>>
>> I'm quite sure, I miss some details here and I do entirely miss the context
>> of why the current approach was chosen, but I hope, we can start a
>> discussion on how to make it possible to reproduce R packages going forward.
> 
> Well, the package names in principle are taken from the metadata contained
> inside the DESCRIPTION file every R package (they use the same name for
> their source - so please do not mix up with Debian package).  There is
> also the concept of versioned dependencies.  We simply check whether this
> package / version is available in Debian.
> 
> I think what would be helpful for me to understand if you could give me
> some example where the same build results in different packages.  I need
> to admit that I might fail to understand the real consequences of the
> code and an example would be pretty helpful, thought.

Sure, I'll try [sorry for the longish introduction but I think it's 
important to get us all on the same page here].

dh-r currently determines Depends/Suggests/Recommends at build time from 
the state of the apt cache.

In a build log, it looks like:

[...]
W: Ignoring specified R dependency: R (>= 2.10)
W: Cannot find a debian package for Suggests:knitr
I: Using r-cran-ggplot2 for Imports:ggplot2
I: Using r-cran-reshape2 for Imports:reshape2
I: BioConductor API version: r-api-bioc-3.19
I: Use r-bioc-qvalue as Debian binary package for variables substitution
[...]

[https://amd64.reproduce.debian.net/api/v0/builds/110379/log]

I'd admit this log is a little special as it has a very sparse apt cache 
containing only the build dependencies of the specific package. (This is 
why it finds corresponding Debian packages for ggplot2, reshape2 but not 
for knitr.)

Looking now at the difference between the package in the Debian archive 
and the one built on reproduce.d.n, we find that the 'Suggests' are missing:

[...]
 Depends: r-api-4.0, r-api-bioc-3.19, r-cran-ggplot2, r-cran-reshape2
-Suggests: r-cran-knitr
 Section: gnu-r
 Priority: optional
[...]

[https://amd64.reproduce.debian.net/api/v0/builds/110379/diffoscope]

So, dh-r drops (at least) missing suggests rather silently at build time.

Imagine now a R package that suggests a R package ('foo') not in Debian 
at the time of uploading to Debian. The result would be that the built 
package would _not_ have a 'Suggests' on r-cran-foo as it doesn't exist 
in Debian a build time.
Fast forwarding to a future Debian where r-cran-foo was uploaded, 
rebuilding the same package (via a binNMU for example) would now lead to 
a different package with 'Suggests: r-cran-foo'.


Hope that helps
Philip

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #64 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Philip Rinn <rinni@inventati.org>

To: Andreas Tille <tille@debian.org>

Cc: debian-r@lists.debian.org, 1089197@bugs.debian.org

Subject: Re: dh-r: Stop generating Depends/Suggests/Recommends from the apt cache

Date: Thu, 19 Dec 2024 13:14:37 +0100

[Message part 1 (text/plain, inline)]

On 19.12.24 at 12:34, Philip Rinn wrote:
> Hi Andreas,
> 
> On 19.12.24 at 11:49, Andreas Tille wrote:
>>> The main idea is to have a mapping table embedded in dh-r which can be
>>> overrode on package level.
>>>
>>> mapping table would basically be
>>>
>>> <R package name> <corresponding Debian package name>
>>> knitr r-cran-knir
>>> ggplot2 r-cran-ggplot2
>>> reshape2 r-cran-reshape2
>>> ...
>>>
>>> containing a mapping for all packages in Debian unstable at the time 
>>> of the
>>> dh-r upload.
>>>
>>> This would of course mean that we'd need to keep that list moderately
>>> updated in the dh-r package.
>>
>> Well, in principle the package *names* can be calculated more or less -
>> we only need to know whether these are CRAN or BioConductor (or other)
>> packages.  However, maintaining dh-r after de facto every new r-*
>> package is a burden I would love to avoid.
> 
> The more I think about it, I guess we should actually _not_ have a list 
> in dh-r but generate the Depends/Suggests/Recommends at the time of 
> creating the _source_ package. So I'd suggest to have what I called 
> 'package level override' in my first mail (but filling d/control 
> directly). [See the package example below for context of why.]
> 
> We'd probably need that list in dh-r for a migration period though.

Ah, this is probably nonsense. We could have a mapping in dh-r plus the 
package specific amends/overrides and still not run into problems like I 
described in my reply.

So to answer your initial remark: As we have package amends/overrides we 
don't need a dh-r upload per new r-* package.
The important property we need to have is that there is a stable (in the 
sense of fixed by the version of dh-r and the Debian package version) 
mapping from R packages to Debian packages.

Best
Philip

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #69 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Jochen Sprickerhof <jspricke@debian.org>

To: Andreas Tille <tille@debian.org>

Cc: Antonio Terceiro <terceiro@debian.org>, 961138@bugs.debian.org, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org, 1089197@bugs.debian.org

Subject: Re: Bug#961138: autodep8 uses host APT packages to generate dependencies for pkg-r-autopkgtest tests

Date: Sun, 2 Mar 2025 08:36:23 +0100

[Message part 1 (text/plain, inline)]

Hi Andreas,

* Andreas Tille <tille@debian.org> [2025-03-01 22:39]:
>if you can provide a patch / MR I'm fine to apply it but currently
>I have no more free spoons to work on this.

The real question here is how to solve #1089197, i.e. how to encode the 
package relationships without depending on the apt cache (grep-aptavail 
or rmadison or anything else that is network related). I have no 
experience with R packages so I would need some input here. Possible 
ways I see are:

1. Don't add relations automatically and let the maintainer do it.

2. Always add all package names from the DISCRIPTION file to Suggest:
   From my understanding this would need some mapping from the R package 
	 name to the Debian package name and maybe a rename of the Debian 
	 packages to simplify the mapping.

3. Add the list of R package in Debian to dh-r and use it to convert the 
   package name. Note that this needs to be updated whenever a R 
	 packages is added or removed in Debian. (This is what dh-python has, 
	 afair).

Cheers Jochen

[signature.asc (application/pgp-signature, inline)]

Message #74 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Antonio Terceiro <terceiro@debian.org>

To: Jochen Sprickerhof <jspricke@debian.org>

Cc: Andreas Tille <tille@debian.org>, 961138@bugs.debian.org, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org, 1089197@bugs.debian.org

Subject: Re: Bug#961138: autodep8 uses host APT packages to generate dependencies for pkg-r-autopkgtest tests

Date: Tue, 4 Mar 2025 16:39:19 -0300

[Message part 1 (text/plain, inline)]

Control: forwarded -1 https://salsa.debian.org/ci-team/autodep8/-/merge_requests/39

On Sun, Mar 02, 2025 at 08:36:23AM +0100, Jochen Sprickerhof wrote:
>Hi Andreas,
>
>* Andreas Tille <tille@debian.org> [2025-03-01 22:39]:
>>if you can provide a patch / MR I'm fine to apply it but currently
>>I have no more free spoons to work on this.
>
>The real question here is how to solve #1089197, i.e. how to encode 
>the package relationships without depending on the apt cache 
>(grep-aptavail or rmadison or anything else that is network related). 
>I have no experience with R packages so I would need some input here. 
>Possible ways I see are:
>
>1. Don't add relations automatically and let the maintainer do it.
>
>2. Always add all package names from the DISCRIPTION file to Suggest:
>   From my understanding this would need some mapping from the R 
>package 	 name to the Debian package name and maybe a rename of the 
>Debian 	 packages to simplify the mapping.
>
>3. Add the list of R package in Debian to dh-r and use it to convert 
>the    package name. Note that this needs to be updated whenever a R 	 
>packages is added or removed in Debian. (This is what dh-python has, 	 
>afair).

From my POV, then only option that is viable in autodep8 is 1, so I'm 
inclined to go with the MR linked above. I understand that at this point 
in the release cycle this could be too intrusive, so I would do that 
after trixie is out.

[signature.asc (application/pgp-signature, inline)]

Message #81 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Andreas Tille <andreas@an3as.eu>

To: Antonio Terceiro <terceiro@debian.org>, Jochen Sprickerhof <jspricke@debian.org>, 961138@bugs.debian.org, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org, 1089197@bugs.debian.org

Subject: Re: Bug#961138: autodep8 uses host APT packages to generate dependencies for pkg-r-autopkgtest tests

Date: Tue, 4 Mar 2025 21:05:54 +0100

Am Tue, Mar 04, 2025 at 04:39:19PM -0300 schrieb Antonio Terceiro:
> 
> From my POV, then only option that is viable in autodep8 is 1, so I'm
> inclined to go with the MR linked above. I understand that at this point in
> the release cycle this could be too intrusive, so I would do that after
> trixie is out.

Thank you for the MR.  I'll leave it unapplied until after the release.

Message #86 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Jochen Sprickerhof <jspricke@debian.org>

To: 1089197@bugs.debian.org, Dirk Eddelbuettel <edd@debian.org>, Dylan Aïssi <dylan.aissi@collabora.com>

Cc: Andreas Tille <tille@debian.org>, Antonio Terceiro <terceiro@debian.org>, 961138@bugs.debian.org, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org

Subject: Re: Bug#1089197: dh-r fails without an apt cache

Date: Sun, 1 Jun 2025 18:08:17 +0200

[Message part 1 (text/plain, inline)]

Hi,

I plan to clean the apt cache by default for forky and implement that 
already in the trixie sbuild. This would make a number of R packages 
(via dh-r) and autodep8 FTBFS. For autodep8 there is an open MR linked 
already and for dh-r a fix has been committed in git:

https://salsa.debian.org/r-pkg-team/dh-r/-/commit/144893ae70c2be8ce403b048b095aabaf6865fd8

Please speak up if there is any problem with this plan. You can test the 
new behaviour with:

sbuild --starting-build-commands='apt-get distclean' -d unstable <pkg>

Note that you may still want to apply more fixes as outlined in my mail 
below.

Cheers Jochen

* Jochen Sprickerhof <jspricke@debian.org> [2025-03-02 08:36]:
>Hi Andreas,
>
>* Andreas Tille <tille@debian.org> [2025-03-01 22:39]:
>>if you can provide a patch / MR I'm fine to apply it but currently
>>I have no more free spoons to work on this.
>
>The real question here is how to solve #1089197, i.e. how to encode 
>the package relationships without depending on the apt cache 
>(grep-aptavail or rmadison or anything else that is network related). 
>I have no experience with R packages so I would need some input here. 
>Possible ways I see are:
>
>1. Don't add relations automatically and let the maintainer do it.
>
>2. Always add all package names from the DISCRIPTION file to Suggest:
>   From my understanding this would need some mapping from the R 
>   package name to the Debian package name and maybe a rename of the 
>   Debian packages to simplify the mapping.
>
>3. Add the list of R package in Debian to dh-r and use it to convert 
>   the package name. Note that this needs to be updated whenever a R 
>   packages is added or removed in Debian. (This is what dh-python has, 
>   afair).
>
>Cheers Jochen

[signature.asc (application/pgp-signature, inline)]

Message #91 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Jochen Sprickerhof <jspricke@debian.org>

To: 961138@bugs.debian.org

Cc: 1089197@bugs.debian.org, Dirk Eddelbuettel <edd@debian.org>, Dylan Aïssi <dylan.aissi@collabora.com>, Andreas Tille <tille@debian.org>, Antonio Terceiro <terceiro@debian.org>, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org

Subject: Re: Bug#961138: Bug#1089197: dh-r fails without an apt cache

Date: Sat, 9 Aug 2025 22:10:44 +0200

[Message part 1 (text/plain, inline)]

Control: severity -1 serious
Control: tags -1 ftbfs

Hi,

trixie is released so marking this RC. Please upload the fix from git.

Cheers Jochen

* Jochen Sprickerhof <jspricke@debian.org> [2025-06-01 18:08]:
>Hi,
>
>I plan to clean the apt cache by default for forky and implement that 
>already in the trixie sbuild. This would make a number of R packages 
>(via dh-r) and autodep8 FTBFS. For autodep8 there is an open MR linked 
>already and for dh-r a fix has been committed in git:
>
>https://salsa.debian.org/r-pkg-team/dh-r/-/commit/144893ae70c2be8ce403b048b095aabaf6865fd8
>
>Please speak up if there is any problem with this plan. You can test 
>the new behaviour with:
>
>sbuild --starting-build-commands='apt-get distclean' -d unstable <pkg>
>
>Note that you may still want to apply more fixes as outlined in my 
>mail below.
>
>Cheers Jochen
>
>* Jochen Sprickerhof <jspricke@debian.org> [2025-03-02 08:36]:
>>Hi Andreas,
>>
>>* Andreas Tille <tille@debian.org> [2025-03-01 22:39]:
>>>if you can provide a patch / MR I'm fine to apply it but currently
>>>I have no more free spoons to work on this.
>>
>>The real question here is how to solve #1089197, i.e. how to encode 
>>the package relationships without depending on the apt cache 
>>(grep-aptavail or rmadison or anything else that is network 
>>related). I have no experience with R packages so I would need some 
>>input here. Possible ways I see are:
>>
>>1. Don't add relations automatically and let the maintainer do it.
>>
>>2. Always add all package names from the DISCRIPTION file to Suggest:
>>  From my understanding this would need some mapping from the R   
>>package name to the Debian package name and maybe a rename of the   
>>Debian packages to simplify the mapping.
>>
>>3. Add the list of R package in Debian to dh-r and use it to convert   
>>the package name. Note that this needs to be updated whenever a R   
>>packages is added or removed in Debian. (This is what dh-python has,   
>>afair).
>>
>>Cheers Jochen
>
>

[signature.asc (application/pgp-signature, inline)]

Message #100 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Jochen Sprickerhof <jspricke@debian.org>

To: 1089197@bugs.debian.org

Cc: Andreas Tille <tille@debian.org>, Antonio Terceiro <terceiro@debian.org>, 961138@bugs.debian.org, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org

Subject: Re: Bug#1089197: Bug#961138: autodep8 uses host APT packages to generate dependencies for pkg-r-autopkgtest tests

Date: Fri, 22 Aug 2025 21:19:38 +0200

[Message part 1 (text/plain, inline)]

Hi Andreas,

as discussed in Brest, I have written a patch to dh-make/update-R to 
hard code the dependencies:

https://salsa.debian.org/r-pkg-team/dh-r/-/merge_requests/9

Note that I don't have any experience with the tooling and only did some 
minimal testing so would appreciate feedback.

Cheers Jochen

* Jochen Sprickerhof <jspricke@debian.org> [2025-03-02 08:36]:
>Hi Andreas,
>
>* Andreas Tille <tille@debian.org> [2025-03-01 22:39]:
>>if you can provide a patch / MR I'm fine to apply it but currently
>>I have no more free spoons to work on this.
>
>The real question here is how to solve #1089197, i.e. how to encode 
>the package relationships without depending on the apt cache 
>(grep-aptavail or rmadison or anything else that is network related). 
>I have no experience with R packages so I would need some input here. 
>Possible ways I see are:
>
>1. Don't add relations automatically and let the maintainer do it.
>
>2. Always add all package names from the DISCRIPTION file to Suggest:
>   From my understanding this would need some mapping from the R 
>package 	 name to the Debian package name and maybe a rename of the 
>Debian 	 packages to simplify the mapping.
>
>3. Add the list of R package in Debian to dh-r and use it to convert 
>the    package name. Note that this needs to be updated whenever a R 	 
>packages is added or removed in Debian. (This is what dh-python has, 	 
>afair).
>
>Cheers Jochen

[signature.asc (application/pgp-signature, inline)]

Message #105 received at 1089197@bugs.debian.org (full text, mbox, reply):

From: Andreas Tille <tille@debian.org>

To: Jochen Sprickerhof <jspricke@debian.org>

Cc: 1089197@bugs.debian.org, Antonio Terceiro <terceiro@debian.org>, 961138@bugs.debian.org, Stefano Rivera <stefanor@debian.org>, debian-ci@lists.debian.org

Subject: Re: Bug#1089197: Bug#961138: autodep8 uses host APT packages to generate dependencies for pkg-r-autopkgtest tests

Date: Fri, 22 Aug 2025 23:00:45 +0200

Hi Jochen,

thanks for the patch.  You might have read that I will not have
time next week but I'll put it on high priority in about 10 days.

Thanks a lot
   Andreas.

Am Fri, Aug 22, 2025 at 09:19:38PM +0200 schrieb Jochen Sprickerhof:
> Hi Andreas,
> 
> as discussed in Brest, I have written a patch to dh-make/update-R to hard
> code the dependencies:
> 
> https://salsa.debian.org/r-pkg-team/dh-r/-/merge_requests/9
> 
> Note that I don't have any experience with the tooling and only did some
> minimal testing so would appreciate feedback.
> 
> Cheers Jochen
> 
> * Jochen Sprickerhof <jspricke@debian.org> [2025-03-02 08:36]:
> > Hi Andreas,
> > 
> > * Andreas Tille <tille@debian.org> [2025-03-01 22:39]:
> > > if you can provide a patch / MR I'm fine to apply it but currently
> > > I have no more free spoons to work on this.
> > 
> > The real question here is how to solve #1089197, i.e. how to encode the
> > package relationships without depending on the apt cache (grep-aptavail
> > or rmadison or anything else that is network related). I have no
> > experience with R packages so I would need some input here. Possible
> > ways I see are:
> > 
> > 1. Don't add relations automatically and let the maintainer do it.
> > 
> > 2. Always add all package names from the DISCRIPTION file to Suggest:
> >   From my understanding this would need some mapping from the R package
> > 	 name to the Debian package name and maybe a rename of the Debian 	
> > packages to simplify the mapping.
> > 
> > 3. Add the list of R package in Debian to dh-r and use it to convert the
> > package name. Note that this needs to be updated whenever a R 	 packages
> > is added or removed in Debian. (This is what dh-python has, 	 afair).
> > 
> > Cheers Jochen
> 
> 



-- 
https://fam-tille.de

Message #108 received at 987714-done@bugs.debian.org (full text, mbox, reply):

From: Paul Gevers <elbrus@debian.org>

To: 987714-done@bugs.debian.org

Subject: Re #987714: Depends/Suggests/Recommends are generate from the apt-cache avail list

Date: Fri, 21 Nov 2025 21:29:05 +0100

[Message part 1 (text/plain, inline)]

Version: 20250812

Hi,

On Tue, 21 Oct 2025 11:39:14 +0200 Andreas Tille <andreas@an3as.eu> wrote:
> what I missed in my last mail: Before we do any large scale uploads we
> should decide about bug #987714 of dh-r.


As I understand it, 
https://salsa.debian.org/r-pkg-team/dh-r/-/commit/144893ae70c2be8ce403b048b095aabaf6865fd8 
was supposed to fix this issue. It was uploaded to unstable on 2025-08-12.

Please reopen if I misinterpreted the history of this bug and the links.

Paul

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Send a report that this bug log contains spam.