Docs for immutable DMs

Describe our existing features better in documentation

.buildkite/pipeline.yaml

@@ -44,3 +44,14 @@ steps:
     plugins:
       - docker#v3.0.1:
           image: "node:10"
+
+  - label: "🌐 i18n"
+    command:
+      - "echo '--- Fetching Dependencies'"
+      - "./scripts/fetch-develop.deps.sh --depth 1"
+      - "yarn install"
+      - "echo '+++ Testing i18n output'"
+      - "yarn diff-i18n"
+    plugins:
+      - docker#v3.0.1:
+          image: "node:10"

.github/ISSUE_TEMPLATE/bug_report.md

@@ -31,9 +31,10 @@ Describe here the problem that you are experiencing, or the feature you are requ
 
 Describe how what happens differs from what you expected.
 
-Log: sent/not sent?    <!-- You can send us the app's logs via the 'Report bug'
-link on the 'Settings' page. Very important for hard-to-reproduce bugs. Please
-file a bug here too! -->
+<!-- Please send us logs for your bug report. They're very important for bugs
+which are hard to reproduce. To do this, create this issue then go to your
+account settings and click 'Submit Debug Logs' from the Help & About tab -->
+Logs being sent: yes/no
 
 <!-- Include screenshots if possible: you can drag and drop images below. -->
 

.github/ISSUE_TEMPLATE/user-interface-or-usability-bug-report.md

@@ -33,9 +33,10 @@ Describe here the problem that you are experiencing, or the feature you are requ
 
 Describe how what happens differs from what you expected.
 
-Log: sent/not sent?    <!-- You can send us the app's logs via the 'Report bug'
-link on the 'Settings' page. Very important for hard-to-reproduce bugs. Please
-file a bug here too! -->
+<!-- Please send us logs for your bug report. They're very important for bugs
+which are hard to reproduce. To do this, create this issue then go to your
+account settings and click 'Submit Debug Logs' from the Help & About tab -->
+Logs being sent: yes/no
 
 <!-- Include screenshots if possible: you can drag and drop images below. -->
 

config.sample.json

@@ -8,7 +8,6 @@
             "base_url": "https://vector.im"
         }
     },
-    "disable_identity_server": false,
     "disable_custom_urls": false,
     "disable_guests": false,
     "disable_login_language_selector": false,
@@ -21,8 +20,10 @@
     "defaultCountryCode": "GB",
     "showLabsSettings": false,
     "features": {
-        "feature_groups": "labs",
-        "feature_pinning": "labs"
+        "feature_pinning": "labs",
+        "feature_custom_status": "labs",
+        "feature_custom_tags": "labs",
+        "feature_state_counters": "labs"
     },
     "default_federate": true,
     "default_theme": "light",

docs/config.md

@@ -20,17 +20,16 @@ For a good example, see https://riot.im/develop/config.json.
      if `default_hs_url` is used.
    * The identity server is used for verifying third party identifiers like emails
      and phone numbers. It is not used to store your password or account information.
-     If not provided, the identity server defaults to vector.im unless `disable_identity_server`
-     is set to true in the config. Currently the only two public identity servers
-     are https://matrix.org and https://vector.im, however in future identity servers
-     will be decentralised.
+     If not provided, the identity server defaults to vector.im. Currently the only 
+     two public identity servers are https://matrix.org and https://vector.im, however 
+     in future identity servers will be decentralised. In the future it will be possible
+     to disable the identity server functionality.
    * Riot will fail to load if a mix of `default_server_config`, `default_server_name`, or
      `default_hs_url` is specified. When multiple sources are specified, it is unclear
      which should take priority and therefore the application cannot continue.
 1. `features`: Lookup of optional features that may be `enable`d, `disable`d, or exposed to the user
    in the `labs` section of settings.  The available optional experimental features vary from
-   release to release. Some of the available features are described in the Labs Feature section
-   of this README.
+   release to release. The available features are described in [labs.md](labs.md).
 1. `showLabsSettings`: Shows the "labs" tab of user settings even when no `features` are enabled
    or present. Useful for getting at settings which may be otherwise hidden.
 1. `brand`: String to pass to your homeserver when configuring email notifications, to let the

docs/labs.md

@@ -1,12 +1,29 @@
 # Labs features
 
 Some notes on the features you can enable by going to `Settings->Labs`. Not exhaustive, chat in
-[#riot-web:matrix.org] for more information.
+[#riot-web:matrix.org](https://matrix.to/#/#riot-web:matrix.org) for more information.
 
 **Be warned! Labs features are not finalised, they may be fragile, they may change, they may be
 dropped. Ask in the room if you are unclear about any details here.**
 
-## Render simple counters in room header
+## Message pinning (`feature_pinning`)
+
+Allows you to pin messages in the room. To pin a message, use the 3 dots to the right of the message
+and select "Pin".
+
+## Custom status (`feature_custom_status`)
+
+An experimental approach for supporting custom status messages across DMs. To set a status, click on
+your avatar next to the message composer.
+
+## Custom tags (`feature_custom_tags`)
+
+An experimental approach for dealing with custom tags. Custom tags will appear in the bottom portion
+of the community filter panel.
+
+Setting custom tags is not supported by Riot.
+
+## Render simple counters in room header (`feature_state_counters`)
 
 Allows rendering of labelled counters above the message list.
 
@@ -29,4 +46,7 @@ Once enabled, send a custom state event to a room to set values:
 
 That's it. Now should see your new counter under the header.
 
-[#riot-web:matrix.org]: https://matrix.to/#/#riot-web:matrix.org
+## Immutable DMs (`feature_immutable_dms`)
+
+Immutable DMs mean you have exactly one direct message with another user, ensuring
+you don't accidentally confuse the other person or yourself.

electron_app/riot.im/README

@@ -1,4 +1,6 @@
 This directory contains the config file for the official riot.im distribution
-of Riot Desktop. You probably do not want to build with this config unless
-you're building the official riot.im distribution, or you'll find your builds
-will replace themselves with the riot.im build.
+of Riot Desktop.
+
+You probably do not want to build with this config unless you're building the
+official riot.im distribution, or you'll find your builds will replace
+themselves with the riot.im build.

electron_app/src/electron-main.js

@@ -1,7 +1,7 @@
 /*
 Copyright 2016 Aviral Dasgupta
 Copyright 2016 OpenMarket Ltd
-Copyright 2018 New Vector Ltd
+Copyright 2018, 2019 New Vector Ltd
 Copyright 2017, 2019 Michael Telatynski <7t3chguy@gmail.com>
 
 Licensed under the Apache License, Version 2.0 (the "License");
@@ -211,7 +211,14 @@ const launcher = new AutoLaunch({
 // work.
 // Also mark it as secure (ie. accessing resources from this
 // protocol and HTTPS won't trigger mixed content warnings).
-protocol.registerStandardSchemes(['vector'], {secure: true});
+protocol.registerSchemesAsPrivileged([{
+    scheme: 'vector',
+    privileges: {
+        standard: true,
+        secure: true,
+        supportFetchAPI: true,
+    },
+}]);
 
 app.on('ready', () => {
     if (argv['devtools']) {

electron_app/src/preload.js

@@ -1,5 +1,5 @@
 /*
-Copyright 2018 New Vector Ltd
+Copyright 2018, 2019 New Vector Ltd
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -14,16 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-const { ipcRenderer, webFrame } = require('electron');
+const { ipcRenderer } = require('electron');
 
 // expose ipcRenderer to the renderer process
 window.ipcRenderer = ipcRenderer;
-
-// Allow the fetch API to load resources from this
-// protocol: this is necessary to load olm.wasm.
-// (Also mark it a secure although we've already
-// done this in the main process).
-webFrame.registerURLSchemeAsPrivileged('vector', {
-    secure: true,
-    supportFetchAPI: true,
-});

package.json

@@ -30,6 +30,7 @@
     "reskindex:watch": "reskindex -h src/header -w",
     "i18n": "matrix-gen-i18n",
     "prunei18n": "matrix-prune-i18n",
+    "diff-i18n": "cp src/i18n/strings/en_EN.json src/i18n/strings/en_EN_orig.json && yarn i18n && node scripts/compare-file.js src/i18n/strings/en_EN_orig.json src/i18n/strings/en_EN.json",
     "build:res": "node scripts/copy-res.js",
     "build:modernizr": "modernizr -c .modernizr.json -d src/vector/modernizr.js",
     "build:compile": "yarn reskindex && babel --source-maps -d lib src",
@@ -44,7 +45,7 @@
     "build": "yarn build:js-sdk && yarn build:react-sdk && yarn reskindex && yarn build:res && yarn build:bundle",
     "build:dev": "yarn build:js-sdk && yarn build:react-sdk && yarn reskindex && yarn build:res && yarn build:bundle:dev",
     "dist": "scripts/package.sh",
-    "install:electron": "install-app-deps",
+    "install:electron": "electron-builder install-app-deps",
     "electron": "yarn install:electron && electron .",
     "start:res": "node scripts/copy-res.js -w",
     "start:js": "webpack-dev-server --host=0.0.0.0 --output-filename=bundles/_dev_/[name].js --output-chunk-filename=bundles/_dev_/[name].js -w --progress --mode development",
@@ -153,7 +154,7 @@
   },
   "build": {
     "appId": "im.riot.app",
-    "electronVersion": "4.2.6",
+    "electronVersion": "5.0.7",
     "files": [
       "node_modules/**",
       "src/**"

riot.im/README

@@ -0,0 +1,11 @@
+This directory contains the config files and deployment scripts for the official
+riot.im distribution of Riot Web at https://riot.im.
+
+You probably do not want to build with this config unless you're building the
+official riot.im distribution, but these files may be useful if you want to
+inspect the configuration used there.
+
+Riot Desktop uses a separate config (see electron_app/riot.im/config.json).
+
+Deployment scripts (such as app/deploy.py) are meant to be run on the web server
+hosting the Riot installation.

riot.im/app/config.json

@@ -0,0 +1,48 @@
+{
+    "default_hs_url": "https://matrix.org",
+    "default_is_url": "https://vector.im",
+    "brand": "Riot",
+    "integrations_ui_url": "https://scalar.vector.im/",
+    "integrations_rest_url": "https://scalar.vector.im/api",
+    "integrations_widgets_urls": [
+        "https://scalar-staging.vector.im/api",
+        "https://scalar.vector.im/api",
+        "https://scalar-staging.riot.im/scalar/api"
+    ],
+    "hosting_signup_link": "https://modular.im/?utm_source=riot-web&utm_medium=web",
+    "bug_report_endpoint_url": "https://riot.im/bugreports/submit",
+    "welcomeUserId": "@riot-bot:matrix.org",
+    "piwik": {
+        "url": "https://piwik.riot.im/",
+        "siteId": 1,
+        "policyUrl": "https://matrix.org/docs/guides/riot_im_cookie_policy"
+    },
+    "phasedRollOut": {
+        "feature_lazyloading": {
+            "offset": 1539684000000,
+            "period": 604800000
+        }
+    },
+    "features": {
+        "feature_lazyloading": "enable",
+        "feature_room_breadcrumbs": "labs"
+    },
+    "roomDirectory": {
+        "servers": [
+            "matrix.org"
+        ]
+    },
+    "enable_presence_by_hs_url": {
+        "https://matrix.org": false
+    },
+    "terms_and_conditions_links": [
+        {
+            "url": "https://riot.im/privacy",
+            "text": "Privacy Policy"
+        },
+        {
+            "url": "https://matrix.org/docs/guides/riot_im_cookie_policy",
+            "text": "Cookie Policy"
+        }
+    ]
+}

riot.im/app/deploy.py

@@ -0,0 +1,191 @@
+#!/usr/bin/env python
+#
+# download and unpack a riot-web tarball.
+#
+# Allows `bundles` to be extracted to a common directory, and a link to
+# config.json to be added.
+
+from __future__ import print_function
+
+import argparse
+import os
+import os.path
+import subprocess
+import sys
+import tarfile
+import shutil
+import glob
+
+try:
+    # python3
+    from urllib.request import urlretrieve
+except ImportError:
+    # python2
+    from urllib import urlretrieve
+
+class DeployException(Exception):
+    pass
+
+def create_relative_symlink(linkname, target):
+    relpath = os.path.relpath(target, os.path.dirname(linkname))
+    print ("Symlink %s -> %s" % (linkname, relpath))
+    os.symlink(relpath, linkname)
+
+
+def move_bundles(source, dest):
+    """Move the contents of the 'bundles' directory to a common dir
+
+    We check that we will not be overwriting anything before we proceed.
+
+    Args:
+        source (str): path to 'bundles' within the extracted tarball
+        dest (str): target common directory
+    """
+
+    if not os.path.isdir(dest):
+        os.mkdir(dest)
+
+    # build a map from source to destination, checking for non-existence as we go.
+    renames = {}
+    for f in os.listdir(source):
+        dst = os.path.join(dest, f)
+        if os.path.exists(dst):
+            print (
+                "Skipping bundle. The bundle includes '%s' which we have previously deployed."
+                % f
+            )
+        else:
+            renames[os.path.join(source, f)] = dst
+
+    for (src, dst) in renames.iteritems():
+        print ("Move %s -> %s" % (src, dst))
+        os.rename(src, dst)
+
+class Deployer:
+    def __init__(self):
+        self.packages_path = "."
+        self.bundles_path = None
+        self.should_clean = False
+        # filename -> symlink path e.g 'config.localhost.json' => '../localhost/config.json'
+        self.symlink_paths = {}
+        self.verify_signature = True
+
+    def deploy(self, tarball, extract_path):
+        """Download a tarball if necessary, and unpack it
+
+        Returns:
+            (str) the path to the unpacked deployment
+        """
+        print("Deploying %s to %s" % (tarball, extract_path))
+
+        name_str = os.path.basename(tarball).replace(".tar.gz", "")
+        extracted_dir = os.path.join(extract_path, name_str)
+        if os.path.exists(extracted_dir):
+            raise DeployException('Cannot unpack %s: %s already exists' % (
+                tarball, extracted_dir))
+
+        downloaded = False
+        if tarball.startswith("http://") or tarball.startswith("https://"):
+            tarball = self.download_and_verify(tarball)
+            print("Downloaded file: %s" % tarball)
+            downloaded = True
+
+        try:
+            with tarfile.open(tarball) as tar:
+                tar.extractall(extract_path)
+        finally:
+            if self.should_clean and downloaded:
+                os.remove(tarball)
+
+        print ("Extracted into: %s" % extracted_dir)
+
+        if self.symlink_paths:
+            for link_path, file_path in self.symlink_paths.iteritems():
+                create_relative_symlink(
+                    target=file_path,
+                    linkname=os.path.join(extracted_dir, link_path)
+                )
+
+        if self.bundles_path:
+            extracted_bundles = os.path.join(extracted_dir, 'bundles')
+            move_bundles(source=extracted_bundles, dest=self.bundles_path)
+
+            # replace the extracted_bundles dir (which may not be empty if some
+            # bundles were skipped) with a symlink to the common dir.
+            shutil.rmtree(extracted_bundles)
+            create_relative_symlink(
+                target=self.bundles_path,
+                linkname=extracted_bundles,
+            )
+        return extracted_dir
+
+    def download_and_verify(self, url):
+        tarball = self.download_file(url)
+
+        if self.verify_signature:
+            sigfile = self.download_file(url + ".asc")
+            subprocess.check_call(["gpg", "--verify", sigfile, tarball])
+
+        return tarball
+
+    def download_file(self, url):
+        if not os.path.isdir(self.packages_path):
+            os.mkdir(self.packages_path)
+        local_filename = os.path.join(self.packages_path,
+                                      url.split('/')[-1])
+        sys.stdout.write("Downloading %s -> %s..." % (url, local_filename))
+        sys.stdout.flush()
+        urlretrieve(url, local_filename)
+        print ("Done")
+        return local_filename
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser("Deploy a Riot build on a web server.")
+    parser.add_argument(
+        "-p", "--packages-dir", default="./packages", help=(
+            "The directory to download the tarball into. (Default: '%(default)s')"
+        )
+    )
+    parser.add_argument(
+        "-e", "--extract-path", default="./deploys", help=(
+            "The location to extract .tar.gz files to. (Default: '%(default)s')"
+        )
+    )
+    parser.add_argument(
+        "-b", "--bundles-dir", nargs='?', default="./bundles", help=(
+            "A directory to move the contents of the 'bundles' directory to. A \
+            symlink to the bundles directory will also be written inside the \
+            extracted tarball. Example: './bundles'. \
+            (Default: '%(default)s')"
+        )
+    )
+    parser.add_argument(
+        "-c", "--clean", action="store_true", default=False, help=(
+            "Remove .tar.gz files after they have been downloaded and extracted. \
+            (Default: %(default)s)"
+        )
+    )
+    parser.add_argument(
+        "--include", nargs='*', default=['./config*.json'], help=(
+            "Symlink these files into the root of the deployed tarball. \
+             Useful for config files and home pages. Supports glob syntax. \
+             (Default: '%(default)s')"
+        )
+    )
+    parser.add_argument(
+        "tarball", help=(
+            "filename of tarball, or URL to download."
+        ),
+    )
+
+    args = parser.parse_args()
+
+    deployer = Deployer()
+    deployer.packages_path = args.packages_dir
+    deployer.bundles_path = args.bundles_dir
+    deployer.should_clean = args.clean
+
+    for include in args.include:
+        deployer.symlink_paths.update({ os.path.basename(pth): pth for pth in glob.iglob(include) })
+
+    deployer.deploy(args.tarball, args.extract_path)

riot.im/develop/config.json

@@ -0,0 +1,54 @@
+{
+    "default_hs_url": "https://matrix.org",
+    "default_is_url": "https://vector.im",
+    "brand": "Riot",
+    "integrations_ui_url": "https://scalar-staging.vector.im/",
+    "integrations_rest_url": "https://scalar-staging.vector.im/api",
+    "integrations_widgets_urls": [
+        "https://scalar-staging.riot.im/scalar/api",
+        "https://scalar-staging.vector.im/api",
+        "https://scalar.vector.im/api"
+    ],
+    "hosting_signup_link": "https://modular.im/?utm_source=riot-web&utm_medium=web",
+    "bug_report_endpoint_url": "https://riot.im/bugreports/submit",
+    "features": {
+        "feature_rich_quoting": "labs",
+        "feature_pinning": "labs",
+        "feature_presence_management": "labs",
+        "feature_sticker_messages": "labs",
+        "feature_jitsi": "labs",
+        "feature_tag_panel": "enable",
+        "feature_keybackup": "labs",
+        "feature_custom_status": "labs",
+        "feature_custom_tags": "labs",
+        "feature_lazyloading": "enable",
+        "feature_tabbed_settings": "labs",
+        "feature_sas": "labs",
+        "feature_room_breadcrumbs": "labs",
+        "feature_state_counters": "labs"
+    },
+    "welcomeUserId": "@riot-bot:matrix.org",
+    "piwik": {
+        "url": "https://piwik.riot.im/",
+        "siteId": 1,
+        "policyUrl": "https://matrix.org/docs/guides/riot_im_cookie_policy"
+    },
+    "roomDirectory": {
+        "servers": [
+            "matrix.org"
+        ]
+    },
+    "enable_presence_by_hs_url": {
+        "https://matrix.org": false
+    },
+    "terms_and_conditions_links": [
+        {
+            "url": "https://riot.im/privacy",
+            "text": "Privacy Policy"
+        },
+        {
+            "url": "https://matrix.org/docs/guides/riot_im_cookie_policy",
+            "text": "Cookie Policy"
+        }
+    ]
+}

scripts/compare-file.js

@@ -0,0 +1,10 @@
+const fs = require("fs");
+
+if (process.argv.length < 4) throw new Error("Missing source and target file arguments");
+
+const sourceFile = fs.readFileSync(process.argv[2], 'utf8');
+const targetFile = fs.readFileSync(process.argv[3], 'utf8');
+
+if (sourceFile !== targetFile) {
+    throw new Error("Files do not match");
+}

src/vector/getconfig.js

@@ -37,7 +37,7 @@ export async function getVectorConfig(relativeLocation) {
 function getConfig(configJsonFilename) {
     return new Promise(function(resolve, reject) {
         request(
-            { method: "GET", url: configJsonFilename },
+            { method: "GET", url: configJsonFilename, qs: { cachebuster: Date.now() } },
             (err, response, body) => {
                 try {
                     if (err || response.status < 200 || response.status >= 300) {

src/vector/rageshakesetup.js

@@ -50,6 +50,10 @@ initRageshake();
 
 global.mxSendRageshake = function(text, withLogs) {
     if (withLogs === undefined) withLogs = true;
+    if (!text || !text.trim()) {
+        console.error("Cannot send a rageshake without a message - please tell us what went wrong");
+        return;
+    }
     require(['matrix-react-sdk/lib/rageshake/submit-rageshake'], (s) => {
         s(SdkConfig.get().bug_report_endpoint_url, {
             userText: text,

test/app-tests/joining.js

@@ -104,7 +104,6 @@ describe('joining a room', function() {
                     hsName: "TEST_ENVIRONMENT",
                     hsNameIsDifferent: false, // yes, we lie
                     isUrl: IS_URL,
-                    identityEnabled: true,
                 }),
             };
 

test/app-tests/loading.js

@@ -153,7 +153,6 @@ describe('loading:', function() {
                 hsName: "TEST_ENVIRONMENT",
                 hsNameIsDifferent: false, // yes, we lie
                 isUrl: DEFAULT_IS_URL,
-                identityEnabled: true,
             }),
             embeddedPages: {
                 homeUrl: 'data:text/html;charset=utf-8;base64,PGh0bWw+PC9odG1sPg==',