projects / pgweb.git / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: 428f299) | patch
Teach search crawler about internal sitemap
authorMagnus Hagander <magnus@hagander.net>
Thu, 23 Mar 2017 15:31:39 +0000 (16:31 +0100)
committerMagnus Hagander <magnus@hagander.net>
Thu, 23 Mar 2017 15:31:39 +0000 (16:31 +0100)
commit7edb14284dba2b521249731073ad0b44267cb479
treee6dcd8235ae07c01e69c18b73399097c8504a216tree
parent428f299f48e4daf507cddfeb520e643c907b3227commit | diff
Teach search crawler about internal sitemap

We only support it for our main website, which uses a sitemap, so
implement it only for that provider. And always probe
sitemap_internal.xml, since we don't even try to access any external
sites on it.
tools/search/crawler/lib/basecrawler.py diff | blob | blame | history
tools/search/crawler/lib/genericsite.py diff | blob | blame | history
tools/search/crawler/lib/sitemapsite.py diff | blob | blame | history
tools/search/sql/schema.sql diff | blob | blame | history
This is the code for the postgresql.org website