Facial recognition has been buzz word lately. It’s been used by technology companies to improve their products, law enforcement to identify criminals, NGOs to identify missing persons, and more. It’s also been used for nefarious reasons, leading to a general negative sentiment in the public eye. In the United States, cities are limiting the use of facial recognition by law enforcement. In the EU, the GDPR explicitly outlines guidelines for it. It has sort of a big brother, Orwellian, feel to it when taken to scale. However, it also provides next-level capabilities in the OSINT community. Previously, many OSINT investigators were limited to results in the form of text. Manual methods allow the analyst to sift through audio, photos, and videos. Now you can use facial recognition to automate a portion of the photo analysis. Let’s take a look at how it’s done.
The first step in this process is building your database of faces. The first thing that comes to mind here is the Faceless God from Game of Thrones. Not actual faces, of course, but photos of individuals you want to use as your constant. In some cases it may be criminals you’re looking to investigate or locate, it could be missing children, or it could even be pictures of yourself. Face_Recognition is a tool that allows you to take that database and compare it to another set of photos.
Let’s say you have frame-by-frames of security footage of a robbery, home surveillance footage, or archives of photos found online from various sources. You can run the face_recognition script to see if any of the photos match the ones in your database. I’ve tested it with photos using various angles as well as of individuals with varying characteristics. It’s surprisingly accurate! It’s also easy to setup and use and easy to use across multiple databases. To properly perform the workflow I will layout throughout this post, go ahead and follow the instructions in the wiki and get this installed now.
For this case study, we are going to scrape photos from Instagram and compare them to a folder of photos we’ve assembled. In order to make this a personalized workflow, go ahead and select an individual you want to identify. This could be an executive, a celebrity, or yourself. Try to pick someone you know you’ll easily find in other sources. The purpose of this is to introduce you to the face_recognition tool and to understand its strengths and weaknesses. We will go over an Instagram scraper tool to extract the second folder containing unknown photos in a minute. Try these examples:
- Melania Trump. We will scrape Donald Trump’s Instagram page and run face_recognition to find photos that contain Melania.
- Kim Kardashian. We will scrape Kanye West’s Instagram page and run face_recognition to find photos that contain Kim.
Feel free to create your own custom use case; however, don’t make it too difficult until you fully understand the input, output, and limitations of the tool.
Instalooter is one of the staples for Instagram scraping in the OSINT community. It’s stable, reliable, and many users can vouch for it. That being said, it does have limitations. Because of Facebook’s general crackdown on scraping, data collection, and data privacy, using non-API methods of collection on any of their platforms is becoming more and more difficult. Instalooter does not use API. There are a few other tools that don’t use API, but I’m using Instalooter as an example because the file format for the output is exactly what we need to use in combination with face_recognition. Other tools like Instaloader exist, but their output is undesirable for this workflow as each photo is downloaded into separate folders with metadata and other information within. We will use Instalooter to scrape all of the photos from target profiles in order to create our variable database. More on that later.
Now that you’ve successfully installed face_recognition and Instalooter, we can begin to discuss this basic workflow for building your very own facial recognition database. Let’s get started.
- Step 1 – Create a folder titled whatever you like. Avoid spaces in the naming convention to repel errors thrown by face_recognition (i.e. source_photos instead of source photos). This folder will contain your constants. If you want to use the examples discussed above, you can insert verified photos of either Melania Trump or Kim Kardashian into the source_photos folder. Make sure to name your photos properly. Use “Melania Trump 1”, “Melania Trump 2”, etc. Face_recognition will give you an output matching to those photo names. Try to use photos using different face angles as well; face_recognition has a good wiki and provided examples of Barack Obama and Joe Biden for reference.
- Step 2 – Choose your target Instagram profile for scraping. If you’re going with Melania Trump, choose Donald Trump’s Instagram. If you’re going Kim K, use Kanye’s Instagram. If you’ve chosen neither, pick a profile closely associated with the individual you’ve entered into your source_photos folder. Once, you’ve chosen your target, run Instalooter to scrape all photos from that profile. They will be saved into a new folder within your directory. Rename that folder unknown_photos.
- Step 3 – Now that we have source photos and unknown photos, we need to run facial recognition comparing the two folders to one another. This can be done by the command face_recognition [insert source_photos directory] [insert unknown_photos directory]. You can find the directory of each folder by simply right clicking the folder and choosing “copy”. Go ahead and run the script and analyze the results. You’ll get options like “no persons found”, “unknown persons found”, or “[match]”.
Now, if you’ve followed all of the above steps, congratulations, you now have a functional, basic facial recognition workflow for OSINT. There are many applications for this process. Law enforcement can use this tool to track criminals or associates of criminals. Counter human traffickers can work with law enforcement to track missing persons or individuals known to be trafficked. Government agencies and private organizations can use this tool to identify persons of interest, or those who pose a potential threat to a VIP. Last, but certainly not in conclusion, social media influencers can use this tool to identify copycats or people stealing their content. What application will you find?
This guide is primarily targeted at OSINT collectors, analysts, and investigators who are not in violation of data privacy restrictions applied to facial recognition. Individuals in the EU are subject to article 9 of clauses within the GDPR restricting facial recognition to any individuals that does not have special access or permission. Also, do not use this tool for black hat methods. I am not advocating the misuse of this tool by any individual. Proceed at your own risk.