Add APIs not listed in the API catalogue

[MansMeg]

There are APIs available that are currently not listed in the package. Several APIs listed here are not in our catalogue (yet): https://www.scb.se/en/services/statistical-programs-for-px-files/px-web/pxweb-examples/

[pitkant]

I went through the list:

PxWeb data bases in some countries/organisations

• Croatian Bureau of Statistics
• Eustat, Basque
• National Statistics, Republic of China (Taiwan) (This seems to be superseded in favour of a homegrown solution)
• National Statistical Service of the Republic of Armenia
• State Statistical Office of the Republic of Macedonia (makstat.stat.gov.mk)
• Statistics and Research Åland (pxweb.asub.ax)
• Statistics Estonia (andmed.stat.ee)
• Statistics Faroe Islands (statbank.hagstova.fo)
• Statistics Finland (statfin.stat.fi)
• Helsinki city (stat.hel.fi)
• Nordstat (Nordic cities) (accessible through stat.hel.fi)
• Statistics Greenland (bank.stat.gl)
• Statistics Iceland (px.hagstofa.is)
• Statistics Latvia (data.stat.gov.lv)
• Statistics Norway (data.ssb.no)
• Statistical Office of the Republic of Slovenia (pxweb.stat.si)
• Statistics Switzerland (www.pxweb.bfs.admin.ch)
• UNECE (w3.unece.org)
• Nordic Statistics database (Nordic Ministries Nordic countries database) (pxweb.nordicstatistics.org)
• Grande Région database (www.grande-region.lu)
• INSTAT, Mali

PxWeb examples in Sweden

• SCB, Statistics Sweden (api.scb.se)
• Jordbruksverket (Swedish Board of Agriculture) (statistik.sjv.se)
• Karlstads kommun (Karlstad municipality)
• Konjunkturinstitutet (National Institute of Economic Research) (statistik.konj.se)
• Linköpings kommun (statistik.linkoping.se)
• Folkhälsomyndigheten (Public Health Agency of Sweden) (fohm-app.folkhalsomyndigheten.se)
• Sundsvalls kommun (Sundsvall municipality)
• Västerås stad (Västerås municipality)
• Västra Götalandsregionen (Region Västra Götaland) (pxwebb2017.vgregion.se)
• Örebro kommun (Örebro municipality)

It is also possible to use components of PxWeb together with selections from an SQL database. (These are maybe not attainable in this package but maybe it's worth exploring)

• Statistics Denmark
• Statistics New Zeeland
• Albania, The Institute of Statistics (INSTAT)

---

I also found some databases done in pxweb that were not in the list above:

• The Generations & Gender Contextual Database
• Philippines Statistics Authority (PSA) OpenSTAT
• Latvian Health Statistics Database
• International Renewable Energy Agency IRENA
• Finnish Centre for Pensions
• INSTITUTO NACIONAL DE ESTATISTICA - Moçambique
• Geostat Statistics Database (Georgia)
• Estonian The Health Statistics and Health Research Database (Andmebaas)
• Nordic Health and Welfare Statistics

Databases listed in the pxweb package catalogue but not included in the list on SCB website:

• The Swedish national institute of economic research, forecast database (prognos.konj.se)
• Statistics Finland (old version) (pxwebapi2.stat.fi)
• LUKE Natural Resources Institute Finland (statdb.luke.fi)
• Verohallinto - Finnish Tax Administration (vero2.stat.fi)
• Icelandic Centre for Retail Studies (px.rsv.is)
• Statistics Moldova (statbank.statistica.md)
• Statistics Kosovo (askdata.rks-gov.net)
• Finnish Transport Safety Agency (trafi2.stat.fi)
• Visit Finland (Rudolf service) (visitfinland.stat.fi)

Other notes:

• Västra Götaland seems to have at least 3 different API urls: https://pxweb2022.vgregion.se, pxwebb2017.vgregion.se and pxwebb2014.vgregion.se. http://www.vgregion.se/statistikdatabas redirects to pxweb2022. All of these, at the time of writing, seem to be out of service so I can't compare them now

[MansMeg]

Great! Could you open up a PR with ehe ones that are missing?

[pitkant]

Entities that did not have API available:

• National Statistical Service of the Republic of Armenia
• Nordic Health and Welfare Statistics

Entities with broken APIs:

• Örebro kommun (old pxweb API version)
• EUSTAT (redirects to net-inter-eustat-45.ejgvdns, reported to their helpdesk)
• INSTAT, Mali (sometimes fails, sometimes produces an error message, sometimes works, unstable internet connection?)
• Icelandic Centre for Retail Studies (px.rsv.is) (no longer existing, timeouts?)

API works fine but produces a warning:

• Sundsvall kommun

Error in curl::curl_fetch_memory(url, handle = handle) : 
  Could not resolve host: net-inter-eustat-45.ejgvdns

###

This is an old PXWEB API not supported by pxweb R package
(version PX-Web 2014 Dec R1 or later is supported): 
http://pxweb.orebro.se/api/v1/sv/

###

1: 34
Error: 
This is an old PXWEB API not supported by pxweb R package
(version PX-Web 2014 Dec R1 or later is supported): 
https://pxweb.instat-mali.com/PXWeb/api/v1/fr/BaseDeDonnees/

1: 34
Error: 
This is not a PXWEB API: 
https://pxweb.instat-mali.com/PXWeb/api/v1/fr/BaseDeDonnees/


==============================================================================
 R PXWEB: Content of 'pxweb.instat-mali.com'
==============================================================================
 [ 1 ] : v1
==============================================================================
Enter your choice:
('esc' = Quit, 'i' = Show id) 

###

http://px.rsv.is/PXWeb/api/v1/en
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [px.rsv.is] Failed to connect to px.rsv.is port 80 after 10009 ms: Timeout was reached

http://px.rsv.is/PXWeb/api/v1/is
Error in curl::curl_fetch_memory(url, handle = handle) : 
  Timeout was reached: [px.rsv.is] Failed to connect to px.rsv.is port 80 after 10004 ms: Timeout was reached


###

Warning message:
PXWEB URL CHANGE:
 from: https://m02-http-pxwebb.login.sundsvall.se/PXWeb_Ext/api/v1/sv/
 to:   https://m02-http-pxwebb.login.sundsvall.se/PXWeb_Ext/api/v1/sv/ 

[MansMeg]

Strange warning from Sundsvall.

I think we can skip the ones that doesnt work. They can still use pxweb but we end up with unnessecary maintenance.