502 Bad Gateway when searching for exploits

[dguido]

From irb with a fresh install of the shodan gem:

>> api = Shodan::WebAPI.new(SHODAN_API_KEY)
=> #>, @base_url="http://www.shodanhq.com/api/", @exploitdb=#>, @api_key="xxx", @msf=#>>
>> api.exploitdb.search('cve-2011-1255')
JSON::ParserError: 757: unexpected token at '

... snipped a bunch of HTML that wouldn't render on GitHub ...

> 

[erran]

The significant portion was: <title>www.shodanhq.com | 502: Bad gateway</title> and the whole error page is in this gist.

As a side note @dguido, html wrapped in triple back ticks ``` would've allowed you to paste it on GitHub (or better yet into a gist).

[dguido]

Ah, yeah I was using pre tags. Thanks for the tip!

[erran]

Are you able to execute a normal search? I'm getting a Net::ReadTimeout exception raised. I can't connect through the URL when used via the browser either. The info function's still working. I'll create a pull request with some error handling for the Shodan gem now. I'll give you a link to install the gem via GitHub (until @achillean releases a new version).

[erran]

Oh, and if you check out @achillean's twitter account. You'll seen there were some problems going on in the last few days.

[dguido]

Regular search works. ExploitDB and MSF search return 502, with both the Python and Ruby libraries.

[achillean]

I rewrote the Exploits search, put it on a separate domain and in the process also normalized the data (so there isn't a need for different classes for ExploitDB and Metasploit). But I didn't yet update the API to make use of the new Exploits website. The new endpoint for the API calls is https://exploits.shodan.io/api and I'm in the process of updating the wrapper classes to use that new URL. Note that the search results now contain the contents of the files, so you don't need to do a separate call anymore to grab/ download the contents of a script.