As part of a new series on the blog, I want to start reaching out to people who normally wouldn’t appear on the podcast and get a little insight on their pathway to OSINT and what they’re working on. This interview is of an OSINT Python developer who made the Scavenger bot. It’s a bot that scrapes PII on paste sites and publishes the results automatically on Twitter as well as locally (if you run the script). I chose to do this interview because I’ve spoken to a lot of OSINT users and developers, but having an OSINT bot was something a bit different. I hope you enjoy this short discussion.
Who are you?
I am just a random security guy who is into hacking and everything around this topic. Like many other people, I typically play CTFs on ctftime.org and do some personal OSINT research in my free time.
How did you get into OSINT?
My way into OSINT was through “classic” hacking activities. As everyone knows reconnaissance is a main part of hacking. Learning as much as you can about your target is important if you want to attack it. When I started diving into the hacking topic about 10 years ago, I soon discovered how interesting it is to gather information from different sources and put them all together to see the whole picture.
What is Scavenger and why did you create it?
Scavenger is a bot which crawls for credential leaks on paste sites like pastebin.com. The idea for this software came to my mind when I searched on pastebin.com and saw that someone uploaded an sql dump of a forum. This caught my attention and I had a look if there are other pastes containing username and password combinations. I discovered that there are a lot but many of them were already deleted by the staff as they contained sensitive information. So it came to my mind to continuously crawl every new created paste and look for sensitive information inside. It turned out that a lot sensitive stuff like config files, credentials, sql dumps and more are uploaded, however sooner or later they get deleted from the servers. So, the bot saves them for later analysis and research, even if the paste gets deleted.
Which paste sites does this bot scrape? Which sites are missing?
At the moment the bot crawls pastebin.com and paste.org. There are a lot of sites missing. Some of them I mention in the readme file of the bot. If you have any suggestions which are worth a look just let me know!
Other than the Scavenger bot on Twitter, what do you do with OSINT?
I am mostly concentrating on Google hacking and Shodan stuff to identify interesting and funny content. At the moment I am also peeking into the onion network to identify the juicy stuff there.
Are there any other OSINT tools you are working on?
I am not really developing other tools. However, I am trying to extend the bot to crawl the onion network and also search for other things other than credentials (config files, private keys and sql dumps).
What are some tips you have for Python developers making their first OSINT tool?
I am not sure if I am the right person to give developers hints because I am a “dirty developer” who produces code which works, but nothing more. However, I want to encourage everyone who is trying to make a tool to publish its code. Even if you think it is not worth publishing, it may help others.
If you want to learn more about this bot, check out the read.me here.
You can also follow the developer on Twitter and ask any questions you have @rnd_infosec_guy
If you enjoyed this interview, make sure to subscribe to my email list. You can let me know what you didn’t like about it by reaching out on Twitter.