A Better Way to Find and Archive OSINT on Twitter
3 min read

A Better Way to Find and Archive OSINT on Twitter

A Better Way to Find and Archive OSINT on Twitter


I was doing some OSINT research on Twitter when I noticed a thread by @fs0c131y discussing an investigation he/she had done into online sellers who are taking advantage of people during the COVID-19 outbreak. It was quite a good thread and is worth checking out but I feel the most interesting and useful part of the post was the first comment I saw. A user named Thorsten G. tweeted “@threadreaderapp unroll” which led to a bot archiving the entire thread and posting it on their website. This got me thinking and I think I may have found a different way of pulling information from Twitter and archiving it.

The Setup

There’s a couple ways you can set this up depending on your investigation. We’ll go through both just in case either applies to your work.

Option 1: Sock Puppet Account

The @threadreaderapp bot only requires you to enter the word “unroll” in the same tweet that you tag their handle in. So you can type in “unroll @threadreaderapp”, “@threadreaderapp unroll”, etc. If you’re using a sock puppet account, you can simple tweet that at the bottom a thread you’re looking into and immediately delete the tweet. The threadreaderapp will still archive the tweet thread. The only risk here is that the user will get a notification of your comment; however, if they go to click on that notification the tweet will no longer exist.

Option 2: threadreaderapp search

If you jumped ahead a bit, you’ll notice that threadreaderapp.com has a search engine on it that allows you to search by account, keyword, or hashtag. This option lets you search for relevant content within its archive, avoiding the risk of tagging a thread yourself. It also allows you to search for otherwise unknown Twitter threads or more outdated content. Best of all, it may have captured threads of Tweets that were deleted by the user so think of this site as another archive site to consider.

Option 3: Twitter Advanced Search

I didn’t originally plan on including this option. I actually only thought of it while writing out the instructions of the previous two options! Another option of locating archived threads is to do an advanced Twitter search for the @threadreaderapp account and include relevant keywords or accounts in your investigation. Additionally, while the message that @threadreaderapp includes in its autogenerated tweet is random, all tweets include the words “the unroll” in them. So if you include that (this exact phrase) in addition to they keyword you’re looking for (any of these words), you should narrow your results down significantly. You can also search for either the combination of “unroll @threadreaderapp” or “@threadreaderapp unroll” to find users who have initiated the archiving process!

Why It’s Better

You may be asking yourself, why don’t you just archive the Twitter thread itself instead of relying on a third party app? To answer that question in a straightforward way, Threadreaderapp has one feature that I really like–it allows you to download videos directly from the thread. Directly on Twitter, you can only copy the link of the video which ultimately only redirects you back to the tweet. Sure, you can use other tools to extract that video, but that doesn’t solve the archiving issue. Threadreaderapp lets you archive the thread AND download all the video content within it.

The Archive

So now that you’ve instructed the bot to archive the Twitter thread, let’s talk about archiving this new page. Sure, the page is technically an archive already, but all of the threads pulled by the bot are possibly subject to be deleted after a certain period of time. Threadreaderapp states in their terms of service:

“Thread Reader reserves the right, without justification or prior notice, to prevent access to any Content or to remove or deactivate any Unroll that may adversely affect the rights of third parties, which you expressly agree.”

So for that reason, we want to create our own archive just incase. I prefer to use Wayback Machine because it will allow you to download the videos from the archive instead of only relying on the original source. If you use archive.today, for example, it won’t save a video file to the archive that is downloadable.

Threadreaderapp example

Wayback Machine example

Archive.today example

I hope you enjoyed this exploration of OSINT on Twitter. This isn’t a ground shaking discovery, but I believe its an improvement to my own process and possible yours too. Feel free to leave a comment or shoot me a DM Twitter if you want to discuss further!

Enjoying these posts? Subscribe for more