[PLUG] W: Duplicate sources.list entry

[JP Vossen <jp@jpsdomain.org>]

Short: when t-shooting "Duplicate sources.list entry" errors make sure 
you consider /etc/apt/source.list.d in addition to /etc/apt/source.list. 
  See grep command below.

______________________
Long:

I was switching from approx to http-replicator on my Intrepid system, 
and I suddenly started getting this error:

Reading package lists... Done
W: Duplicate sources.list entry http://packages.medibuntu.org 
intrepid/free Packages 
(/var/lib/apt/lists/packages.medibuntu.org_dists_intrepid_free_binary-i386_Packages)
W: Duplicate sources.list entry http://packages.medibuntu.org 
intrepid/non-free Packages 
(/var/lib/apt/lists/packages.medibuntu.org_dists_intrepid_non-free_binary-i386_Packages)
W: You may want to run apt-get update to correct these problems


My reading of that matches up with everything I can find in Google, 
which says to remove the duplicate entry in /etc/apt/sources.list.  The 
only problem is that there is no duplicate:

	# grep -i medi sources.list
	deb http://packages.medibuntu.org/  intrepid  free non-free


Since I was goofing with APT proxy servers I tried deleting local 
/var/lib/apt/lists/*medi* and the appropriate files on the proxy but 
that doesn't help.

I finally remembered to check /etc/apt/source.list.d, and there it was: 
medibuntu.list.  I'm not quite sure how that got there, since AFAICT 
it's not owned by any package and I don't remember putting it there. 
But I deleted it and now the error is gone.

	# grep -R '^deb' sources.list sources.list.d | grep 'medi'
	sources.list:deb http://packages.medibuntu.org/       intrepid 
   free non-free
	sources.list.d/medibuntu.list:deb http://packages.medibuntu.org/ 
intrepid free non-free


The confusing thing was that I only got the error when using 
http-replicator; not when using approx or no proxy.  That happened 
because approx requires re-written sources.list files, so there was the 
sources.list.d.medibuntu.list version and the re-written one so they 
were different.  I don't know why I did not get the error when not using 
the apt.conf "Acquire::http::Proxy" line though.


As I just said approx requires re-written sources.list lines like:
	deb http://aptproxy:9999/medibuntu/ intrepid  free non-free

In my particular circumstances that's very annoying for the following 
reasons, and is why I am trying http-replicator:

1) You need to totally re-write your source.list, the stock one won't work.
2) You need to totally re-write your source.list when switching between 
your LAN (with your proxy) and elsewhere (no proxy).
3) You must define each "source" in approx.conf on the proxy itself, you 
can't just add it to sources.list on your client and have it work.
4) One implication of #3 is that you can't have duplicate sources, so 
you need to re-write even more stuff in source.list.  For example, you 
can't do this in the proxy's approx.conf:
	security       http://security.debian.org/debian-security
	security       http://security.ubuntu.com/ubuntu
5) Approx sort-of works as a regular proxy, so you can use 
"http://aptproxy:9999/"; as the proxy server while doing a 'Net install 
of Debian, but another implication of #3/4 is that it will then not be 
able to find security updates, which defeats some of the purpose of 
doing a 'Net install in the first place.


http-replicator not only Just Works and solves all of the above, but it 
can be used by RPM and probably even Windows Update (not tested). 
However, it isn't perfect either.

1) It looks like it may be abandoned upstream.
2) The .deb packages are seriously broken, and only install it about 
half-way.  There area a bunch of manual steps required to make it 
actually work after that (https://help.ubuntu.com/community/HttpReplicator).
3) Its idea of cleaning up after itself is to only keep the most recent 
version of a package.  That's a good start, but will eventually leave 1 
of each package you cached from old versions, as you upgrade.  As far as 
I can tell there is no time-based cleanup.

Once installed, the key to using it is to add a line like this to 
/etc/apt/apt.conf:
	Acquire::http::Proxy "http://aptproxy:9998";;
	
(I have/had approx on :9999, so http-replicator is on :9998.  Its 
default is actually 8080.)


Finally, I want to say that I like approx quite a bit, it's just that my 
needs are evolving away from the problems it solves, a bit.

Hopefully this will be useful to someone...
JP
----------------------------|:::======|-------------------------------
JP Vossen, CISSP            |:::======|      http://bashcookbook.com/
My Account, My Opinions     |=========|      http://www.jpsdomain.org/
----------------------------|=========|-------------------------------
"Microsoft Tax" = the additional hardware & yearly fees for the add-on
software required to protect Windows from its own poorly designed and
implemented self, while the overhead incidentally flattens Moore's Law.
___________________________________________________________________________
Philadelphia Linux Users Group         --        http://www.phillylinux.org
Announcements - http://lists.phillylinux.org/mailman/listinfo/plug-announce
General Discussion  --   http://lists.phillylinux.org/mailman/listinfo/plug