FlowCrypt Project Structure and Overview

These notes are extracted from an email, so they lack some context. Still, they may be useful to first-time contributors, and should probably be a required reading if you are looking to contribute.

Entrypoint, where to start

There's no entry point. Or rather, there are 50 entry points, depending what you're looking at.

Note: no MVC here, though I'm not ideologically opposed. Just not implemented that way now.

Extension pages

Each page has it's own .ts file with the same name. Es an example, examine the html import tags in extension/chrome/settings/modules/help.htm. The last tag is the interesting one:

<script src="help.js" type="module"></script>

In this case help.js in the same folder (same name, same folder by convention). That's your entry point. type="module" Tells the browser to treat it as ES6 module. help.js is the name after TypeScript is transpiled. Source code for this file will end with .ts. So help.ts is your entry point for the help page.

Continuing the thread, help.ts loads a bunch of deps as es6 modules on top:

import { VERSION } from '../../../js/common/core/const.js';
import { Catch } from '../../../js/common/platform/catch.js';
import { Str } from '../../../js/common/core/common.js';
import { Xss, Ui, Env } from '../../../js/common/browser.js';
import { BrowserMsg } from '../../../js/common/extension.js';
import { Api } from '../../../js/common/api/api.js';

Note: We don't transpile ES6 modules away, instead we declare in manifest that we only support browsers that do es6 modules natively.

Then there is a big asynch/catch block, all entry points (pages) have one. This is to catch and report any errors that happen during initialization, plus it gives us a way to use the await keyword throughout initialization.

Catch.try(async () => {

  ...

})();

Next we check url parameters. Notice how we never use uncheckedUrlParams in the code below this, all input parameters should be somehow checked first. The project follows a fail fast and loudly philosophy. Any deviation from expected input should throw early, so that we know where to fix the problem when it surfaces.

  const uncheckedUrlParams = Env.urlParams(['acctEmail', 'parentTabId', 'bugReport']);
  const acctEmail = Env.urlParamRequire.optionalString(uncheckedUrlParams, 'acctEmail');
  const parentTabId = Env.urlParamRequire.string(uncheckedUrlParams, 'parentTabId');
  const bugReport = Env.urlParamRequire.optionalString(uncheckedUrlParams, 'bugReport');

Sometimes page may need something from storage, which we may want to load on the top. Our example help.ts page does not have any storage needs, but if it did, it would look something like this:

  // load primary private key - example
  const [primaryKi] = await Store.keysGet(acctEmail, ['primary']);
  Ui.abortAndRenderErrorIfKeyinfoEmpty(primaryKi);

  // load some other stuff from local storage
  const { addresses } = await Store.getAcct(acctEmail, ['addresses']);

Notice how we failed fast and loudly there with Ui.abortAndRenderErrorIfKeyinfoEmpty. We don't always have to do this, because our code would get quite cluttered if we did. But for things that come from external sources (not generated by our own currently running code), we check them and fail fast. This includes local storage/database as well as page url parameters.

Then comes app logic. Probably you declare a few functions, then bind functions to button clicks on the bottom. This part is mostly free-form, and would probably benefit from better structure across the project.

Background page

Background tasks happen here. Certain browser actions are only allowed from a background page. You'll find the background page at extension/js/background_page/background_page.htm, which leads you to background_page.ts in the same folder.

Content scripts

This is trickier. If you are an experienced extension developer, your intuition would first bring you to extension/manifest.js to find this:

  "content_scripts": [
    {
      "matches": [
        "*://mail.google.com/*",
        "*://inbox.google.com/*"
      ],
      "css": [
        "/css/webmail.css"
      ],
      "js": [
        "/lib/purify.min.js",
        "/lib/jquery.min.js",
        "/lib/openpgp.js",
        "/js/content_scripts/webmail_bundle.js"
      ]
    },

The purify, jquery and openpgp are dependencies and extension/js/content_scripts/webmail_bundle.js is our bundled code for content scripts (browsers don't yet support ES6 import syntax in content script, so we need to have a mechanism to bundle our beautiful ES6 modules into... webmail_bundle. This is done in tooling/bundle-content-scripts.ts (a Nodejs script). Custom stuff. You may think that Babel and/or Webpack would be perfect for this, and you may be right. But I wasn't able to set it up without it transpiling our ES6+ code into a pile of garbage. We run exclusively on modern browsers, so we don't need the transpile (other than from TypeScript to modern JavaScript). Result:

buildContentScript(([] as string[]).concat(
  getFilesInDir(`${sourceDir}/common/platform`, /\.js$/, false),
  getFilesInDir(`${sourceDir}/common/core`, /\.js$/, false),
  getFilesInDir(`${sourceDir}/common/api`, /\.js$/, false),
  getFilesInDir(`${sourceDir}/common`, /\.js$/, false),
  getFilesInDir(`${sourceDir}/content_scripts/webmail`, /\.js$/),
), 'webmail_bundle.js');

By (our) convention, the last js/ts file tends to be the entry point and everything else tend to be libraries / dependencies. So there you have it, quite hidden, extension/js/content_scripts/webmail/webmail.ts is your entrypoint for webmail related content scripts (stuff that runs on mail.google.com and such).

Note: now that I'm reviewing it, it seems to include files into the bundle alphabetically from each folder. That works since the entrypoint is webmail.ts, and the letter w is almost at the end of the alphabet.. this could be improved. Because this code doesn't change much, for now I'm leaving it as is.

Where to put general code

Shared code that you expect to reuse across the project would go somewhere on the extension/js/common folder, as a method of an appropriate class, or a new class.

Note: I'm open to have one big class per file in the future. For now you may find several semi-related classes in one file.

Development workflow

Assuming you use vscode, here's what I do: I code until I'm bored and then I hit F5. F5 in my vscode is mapped to build task which is mapped to npm run-script build. I have a browser open on second monitor and I hit F5 there too and a see the new thing.

Some people prefer to use a live reload mechanism after each change. You can check something like that into the repo if it's non intrusive, eg as a new npm script.

Project dependencies

You can mostly ignore dependencies listed in package.json - they are for development only. Actual extension dependencies are checked into the repo in extension/lib.

There you'll find, among others:

• bootstrap - not used much, mostly used in extension/chrome/settings/index.htm
• featherlight - used as a lightbox to render pages in settings
• fine-uploader - user file uploads (compose box)