In-App Notification Channel
Milestone ID: 4
In-App Notification Channel
Canonical Document: https://docs.google.com/document/d/1exFXiXGI5__20u15hV2UF5OaZ47qWF6SKIzREtyJG-E/edit
Contacts: Jean-Paul Calderone (for now)
Date: 2019-03-27
This is a new feature of the PrivateStorage.io grid in conjunction with the PrivateStorage client which allows infrequent, one-to-all broadcasting of small amounts of plaintext from the PrivateStorage.io grid operators to all of its customers. This is intended first to enable in-app notification of new PrivateStorage client releases and potentially deliver a warning when the beta period is about to end.
Rationale
We are performing frequent PrivateStorage client releases to improve overall functionality and usability of the service. We want all users of the grid to have the opportunity to run the latest version of the software during the beta so that when we gather feedback from them we receive information about experiences with the latest version of the service we have attempted to deliver. We also just want them to have the best experience possible during the beta, including using all of the latest features we develop, avoiding any bugs we've already fixed, and not being vulnerable to any security issues we are aware of.
After the beta period, we also anticipate wanting to make it as easy as possible to run the latest client to receive the best user experience.
Delivering this information in-app creates the opportunity to preserve more of the privacy-oriented user experience that is a central goal of the service. It will allow us to avoid maintaining an email list to deliver email-based notifications, it will allow users to avoid the tedious and potentially compromising task of manually monitoring our website for new releases, and it will remove the growing burden on the customer support team to individually notify users of new releases.
We are taking this on now to test the ideas that in-app notifications satisfy user desires and that we can create a satisfactory user experience.
Personas
- End user: A participant in the PrivateStorage.io beta program who downloads the PrivateStorage.io client and uses it to interact with the storage grid.
- Product manager: Any member of PrivateStorage.io staff who is responsible for the user-facing content that makes up a PrivateStorage client release notifications.
- Notification engineer: Any member of PrivateStorage.io staff who is responsible for publishing release notifications to end users.
User stories
Consume Release Notification
As an end user, I want to know when there is a release of the PrivateStorage client available so that I can upgrade to it.
Acceptance Criteria
- Users receive notifications automatically so long as the PrivateStorage client is running and no additional actions are required on their part.
- The confidentiality and integrity guarantees are at least as great as those provided by the PrivateStorage client for user data.
- The privacy properties of the PrivateStorage client are preserved by the notification system (eg, reading a notification does not directly contact a third-party service).
- A notification is shown once to the user.
Author Release Notification
As a product manager, I want to be able to specify the content of a release notification as a simple text file so that I can easily create the content and then communicate it precisely to a notification engineer for publication.
Acceptance Criteria
(couldn't think of any)
Publish Release Notification
As a notification engineer, I want to be able to accept notification content from a product manager as a plaintext file and publish it so that end users receive the desired content.
Acceptance Criteria
- It must be possible to publish notifications to the staging grid without publishing them to the production grid to QA the notification content without exposing it to users.
- Publishing a new notification cannot remove old notifications from the system.
Constraints and requirements
Must
Use of the notification system must preserve the privacy properties of the PrivateStorage client application.
Notifications must be delivered to end users on average within 12 hours (as long as the end user is running the client).
Notifications must be visible somewhere in the PrivateStorage.io client GUI.
Storage costs of notifications on the grid must scale with less than O(number of end users).
Published notifications must be immutable except for possible eventual deletion.
The authority to publish notifications must be limited to the notification engineer(ing team).
Nice to have
End users can browse old notifications.
End users can dismiss a notification without taking action and be reminded of it later.
End users are informed of the presence of a notification via some persistent state (eg a system tray icon change or a system notification message).
The authority to publish notifications can be distributed across a team to eliminate a single point of failure (either a notification engineer acting in bad faith or an accident such as key material loss).
Messages can be retrieved from the grid by end users in a staggered fashion to distribute the load evenly.
End users are not interrupted by a notification dialog pop-up when they are not using the PrivateStorage client (if they receive as system notification message instead).
Must not
Consumption of notifications by end users must not reveal more personally identifying information than is revealed by use of the system without this notifications feature.
Out of scope
Notifications do not need to be delivered to end users if they are not running the client.
Notifications do not need to be localized (see separate localization design document todo).
Notifications do not need to provide clickable links to download artifacts.
Notifications do not need to support rich presentation (eg html).
It does not need to be possible to edit or delete published notifications.
It does not need to be possible for users to delay or reschedule the notification.
Success
How will we know when we are done?
The client delivers a notification from the PS team to the end user to let them know there is an updated version available.
How will we measure how well we have done?
We will survey users during beta testing to see how they like it and use it. We can also engage with users in forum discussions and targeted user testing.
Alternatives considered
- Current solution: A customer support representative can email each user when a new release is available. This involves minimal engineering effort but ongoing customer support effort. It also complicates opt-out compared to the proposed design. Users need to negotiate with customer support to stop receiving emails instead of just stop using the PrivateStorage client. It also involves an ongoing email-based relationship which is only acceptable during the beta period. Post-beta, we will have no email-based relationship with end users.
- Requiring an email address (personal data) for updates notifications is not ideal and not a requirement.
- An announcement can be posted on the website. This involves minimal engineering effort but ongoing end user effort and diligence to check periodically check the website for notifications of a new release. It allows much more flexibility (presentation of any web content) on our part. It exposes the end
- This requires users to know to visit the website at regular intervals. Push notifications to users is better, especially for security updates.
- A notification feed (RSS) can be posted on the website. This involves minimal engineering effort but requires the end user to operate an RSS client in addition to the PrivateStorage client. RSS activity to PrivateStorage servers may be de-anonymizing.
Detailed design
Focus on external and internal interfaces, how externally-triggered system events (e.g. sudden reboot; network congestion) will affect the system, scalability and performance.
Notification content will be placed onto the PrivateStorage grid itself. The PrivateStorage client is already configured to be able to retrieve content from this grid and doing so introduces no new kinds of data or metadata about the end user.
Backend
The notification engineer will create a new directory on the PrivateStorage grid1, the notification root. The read-cap for this directory will be added to the grid description used to configure the PrivateStorage client (baked in during the beta period, possibly transferred via magic-wormhole in the future). The notification engineer will secure the write-cap for this directory2.
The notification engineer will create a single sub-directory in this directory named "v1
", the v1 notification directory3.
To publish a new announcement, the notification engineer will add a new file to the v1
directory. The file is named ISO8601-TIMESTAMP.txt4. The contents of the file are exactly the contents provided to the notification engineer by the product manager.
Frontend
The PrivateStorage client will poll the notification root to find the v1 notification directory. It will ignore any other contents it finds (until a future notification spec says otherwise). Once it finds the v1 notification directory it will poll it for files. Polling frequency is described below. When a previously unseen file is observed, it is downloaded and an alert is generated for the user with its content, then the file is marked as processed so it is ignored in the future.
Data integrity
Integrity of the notifications themselves is managed completely by Tahoe-LAFS. Therefore, it is expected that the necessary and ongoing maintenance to maintain the Tahoe-LAFS storage grid is the only operational requirement to ensure the integrity of the notifications.
Integrity of the writecap is a separate concern. The writecap is the single point-of-failure for the notification system. Integrity of the writecap will be maintained by preserving an unencrypted copy on the notification publication host and an additional encrypted5 copy (a backup) on a dedicated removable USB flash storage device which is stored in a distinct physical enclave.
Security
Overview / Identification
The notification system represents a means to communicate with all PrivateStorage end users in a way that lends a high degree of authority to the communications. It is important that only communication supporting the legitimate business of PrivateStorage.io be allowed over the notification system.
Threats
An adversary targeting a specific PrivateStorage.io end user may seek to abuse the notification channel to compromise that end user. Such an adversary may be able to observe operation of either the service, the end user, or both. They may also seek network access to systems operated by PrivateStorage.io, the end user, or both (for example, by leveraging exploits in software in those systems, not necessarily software provided by PrivateStorage.io for the purposes of using the service).
Vulnerabilities
When the notification engineer operates Tahoe-LAFS on the notification host to publish a new notification, the writecap becomes readable on a network-attached host. If the notification host is compromised in a way which allows it to be read and exfiltrated, security of the notification system is completely compromised.
Risk
If the writecap is compromised, it becomes possible for the adversary to generate arbitrary notifications to end users. Such notifications could be used to launch a denial of service attack against users. For example, a large volume of notifications could be generated so that the PrivateStorage client exhausts CPU, network, or memory resources of the client host. Alternatively, notifications could be generated such that the user is constantly annoyed by meaningless notifications. Apart from the annoyance, this would destroy the value of the channel for legitimate uses by causing the user to disregard it.
Application
Actions that can be taken to improve security in this area include:
- The writecap can be closely guarded.
- It can be protected by hardware security (eg, locked against use without the presence of a hardware security device such as a yuibkey).
- It can be protected by software security (eg, it can reside only on encrypted storage).
- It can be protected by physical security (eg, placed inside a physical lock box).
- It can be kept air-gapped at all times (or to the greatest extent possible).
- The PrivateStorage client can limit the number and size of notifications it will retrieve at any one time.
- The PrivateStorage client can limit the number of notifications it will generate to draw the end user's attention.
- Notification content itself can be signed by a separate key which is subject to a distinct security regime. The PrivateStorage client can verify these signatures before raising a notification to the end user.
The countermeasures that can be taken to improve security in this area include:
- The existing notifications can be monitored and verified against the expected legitimate notifications to detect a compromise
The counteranalysis which can be performed to improve security in this area includes:
- The role of notification engineer could be replicated. Each notification engineer could publish each notification. Only one notification engineer would hold the true writecap. An adversary would have to compromise multiple writecaps or first determine which notification engineer holds the true writecap.
Backwards compatibility
A future version of this notification channel may require changes to the structure of notifications themselves. In this case, compatibility is preserved with current versions of the PrivateStorage client by continuing to publish v1 compatible messages according to the current scheme. Notifications which conform to a different specification are published to a new vN directory where N is greater than 1. Future versions of the PrivateStorage client are required to interpret such messages.
The reverse scenario is not envisioned. It is expected that at all times the notification system deployed on the PrivateStorage grid will be as new as or newer than the version deployed in all PrivateStorage clients.
Performance and scalability
The current design goals are to support the beta testing period. We expect to have 1000 participants in the beta test.
The load associated with notification system polling is expected to be dwarfed by the load associated with magic folder polling. Each magic folder incurs a poll every 10 minutes. If the notification system incurs a poll every 12 hours then it contributes 1/72 as much load to the system as magic folders polling contributes. Put another way, if a grid can support a maximum of 1000 magic folder users without notifications then it should support a maximum of 986 magic folder users with notifications.
With naive share placement, all clients will poll the same storage server (or servers) for notifications. This does mean that all additional notification system load is incurred by a fraction of the overall storage grid. Considering the above, this may mean that it has difficulty operating on grids scaled beyond 72 servers (because when you have 72 storage servers you expect 1/72 of overall grid load to be as much as any single server can handle on its own and so when you have more than 72 storage servers this is more load than any single server can handle; hypothetically: I haven't measured).
Scalability may be improved by changing the Tahoe-LAFS download algorithm to add variability to server selection for share retrieval.6
The load associated with the publication of new notifications is negligible in the extreme.
Further reading
Notes
-
tahoe create-alias privatestorage-notifications-production:
↩ -
The entire client used by the notification engineer will live on a managed, dedicated host which is disconnected and powered off when not in use. This is the notification host. All future interactions that require the notification writecap will take place on the notification host.
↩ -
tahoe mkdir privatestorage-notifications-production:v1
↩ -
tahoe cp release-v0.4.txt privatestorage-notifications-production:v1/$(date -Iseconds).txt
↩ -
Clients permute the connect servers list to determine an order in which to ask servers for shares. All clients permute this list in exactly the same way. A readcap contains within it the total number of shares that exist (N). Clients could use a client-private value to permute the first N servers of the permuted list. This produces an ordering of N servers that is diverse across different clients but which still gives all clients a first k (number of shares needed; k <= N) servers which are highly likely to hold a share.
↩