Concepts #

Offline Twitter is built on certain foundational concepts, which are helpful to understand in order to maximize your use of the application.

Profile directory #

A “profile directory” is a folder on your computer that stores a self-contained repository of Twitter data. This includes users, tweets, interactions, images and videos, followers, lists, DMs, and more.

Sessions #

A “session” represents a user who is logged in (i.e., you). Each session is saved in the profile directory. When you log into Twitter through the app, a session is created and saved in a file called [your-user-handle].session. This session is used to authenticate your scrape requests to twitter.com. It is also how the web UI tracks who you are currently logged in as.

Don’t share a .session file with anyone you don’t trust, as it contains an authentication cookie and can therefore be used to impersonate you on Twitter.

Login sessions on twitter.com have a near-indefinite lifetime, so after you log in once, the same session could be valid for months. Offline Twitter doesn’t make you enter your login information again if you already have a session, you can reuse the same session for as long as it remains valid.

When using the CLI, the --session [your-session] flag is required for most scraping operations.

Multiple sessions #

You can have multiple accounts logged in at the same time. Each account will get its own session. In the web UI you can switch between active accounts on the Login page.

Scraping vs reloading #

Offline Twitter lets you build a permanent archive of tweets and other content from Twitter. Scraping is the process of fetching data from Twitter and saving it in a profile directory.

When browsing, by default, Offline Twitter is lazy; it only scrapes when necessary. This makes browsing and page load times faster. However, it means that reloading a page does not necessarily re-scrape it! Every scrapeable item has a “refresh” icon in the corner, which you can click to force a re-scrape.

When a scrape is occurring, a spinner will appear over the thing that’s being scraped.

Composability #

A profile directory contains a SQLite database file called twitter.db. It contains all the data, except for media files. SQLite is an open format, which means it’s easy to use other programs on the data that Offline Twitter scrapes. See the Integrations page for an example of this.