Many of those conducting OSINT use very basic methods to extract information. Usually, they are limited to tools that have a graphic user interface (GUI). Many tools, like TweetBeaver, who use a graphic user interface are very powerful. This post isn’t to discredit any tools that use a GUI, its intent is to introduce users who rely on a tool with a GUI to tools that require a little more effort on the user end, but yield much more customizable results. Let’s take a look at 2 OSINT tools that produce advanced results using intermediate computer knowledge.
Photon is self-described as an “Incredibly fast crawler designed for recon.” That being said, the term “recon” suggests this is a tool used for penetration testing. Now, before all of my traditional OSINT guys and gals head for the hills, there are many intelligence applications for Photon beyond penetration testing. Here’s what it can do.
It can pull all internal and external links from a website in seconds
If you’re wondering whether a website is linked to another or references certain material, find out in seconds. If you’re wondering how many broken links are on a website (find out if a web site is actively managed), find out in seconds by running the results through a broken link checker. If you’d like to monitor change over time, run Photon today and then run it again in three months, measure the difference.
It can pull all emails, social media accounts, and amazon buckets in seconds
This is self explanatory. IF YOU’RE CONDUCTING AN INVESTIGATION, RUN PHOTON! You can find out all points of contact from a website with a simple query. If you’re doing a social engineering audit, extract all of your targets in seconds. If you’re trying to measure change over time, including staff changes, social media additions, or company information (buckets), run Photon today and again in six months. There are so many applications with this intel.
It can extract all files including PDF, PNG, XML, etc
You can Google Dork with “site:” and “filetype:” if you want. That will let you view in your search engine all the PDF, PNG, or XML files a URL hosts. You can then click through all of the files individually, save them, organize a folder, etc. Or you can just run Photon, target a URL, and let it do all the work for you. Your choice.
How to use it
Don’t be scared by running manual commands to your computer. Once you do it 10-15 times, you won’t want to do anything else. Here’s what you need to run Photon.
- Install Python
- Install Git
- Use Git to clone Photon by entering this into your command line (git clone https://github.com/s0md3v/Photon.git). You can also find the “https://github.com/s0md3v/Photon.git” I entered by clicking the “Clone or download” button on the Github page.
- Run these commands: “cd Photon” then “pip install -r requirements.txt”
- View the Wiki
- Run these commands: “cd Photon” then “python photon.py -u “https://urlofchoice.com”
- View the results in the Photon folder
7 steps people. It’s that easy. Most of these you’ve already done if you’ve been curious enough.
GasMask is self described as an “All in one Information gathering tool – OSINT”. It’s made by twelvesec and is very powerful. Once again, the target audience for GasMask is penetration testers, if you’re a pen tester, this is perfect for you. If not, don’t be discouraged. The GitHub page isn’t as robust at Photon’s, but it’s still self explanatory and if you’ve figured out how to use Photon, then this process is virtually identical. So what can GasMask do? Here’s a list.
Multiple index searching
You can run a domain name through the following indexes simultaneously:
Need I say more? You can even interact with Shodan using the Shodan API key.
What’s this useful for (minus penetration testing)?
If you’re conducting an investigation and you find that your target has a radical blog, you want to see if it’s indexed anywhere else online (including social media), plug it into GasMask. If you’ve found a website hosting illegal information, you can track where else it is found online using GasMask. If you’re trying to track the spread of propaganda, you can run a misinformation site through GasMask. If you’re trying to hunt down a cybersquatting or typosquatting user, run the domain through GasMask.
Here’s a video with a small walkthrough (mute the awful music).
If you want to learn more, visit their GitHub page.
I’ll be doing a few more of these in the future as I discover more tools that are valuable for OSINT. As always, I’m trying to create a bridge between the infosec and physec community by bringing terminology from the physical world to the information world and vice versa. I don’t consider myself an infosec guy, but I understand enough of the lingo to at least function as a liaison between the too. Please reach out to me on Twitter for any questions or suggestions. Also, check the podcast @osintpodcast on Twitter or just go to osintpodcast.com