Pet Environment Feature for Multipass ===================================== Dev Impacts, Plans, and Decisions --------------------------------- ### Notice This draft is based on my current understanding, which may well be limited/incomplete/wrong. It presents subjective recommendations and opinions which may be flawed and/or subject to change. ### Contents * [Notice](#notice) * [Contents](#contents) * [General Goal](#general-goal) * [Scope](#scope) * [Baseline](#baseline) * [Daemon impacts](#daemon-impacts) - [Primary instance lifetime](#primary-instance-lifetime) + [Creation](#creation) * [Feedback to the user](#feedback-to-the-user) * [Create vs Launch](#create-vs-launch) + [Destruction](#destruction) - [Set primary](#set-primary) + [launch as primary](#launch-as-primary) + [Unset primary](#unset-primary) + [Primary name vs attribute](#primary-name-vs-attribute) - [Default target instance](#default-target-instance) - [Persistifying](#persistifying) - [Canceling a start-up](#canceling-a-start-up) * [CLI client impacts](#cli-client-impacts) - [Existing commands](#existing-commands) + [`--help`](#---help-) + [`--verbose`](#---verbose-) + [`copy-files`](#-copy-files-) + [`delete`](#-delete-) + [`exec`](#-exec-) + [`find`](#-find-) + [`help`](#-help-) + [`info`](#-info-) + [`launch`](#-launch-) + [`list`](#-list-) + [`mount`](#-mount-) + [`purge`](#-purge-) + [`recover`](#-recover-) + [`restart`](#-restart-) + [`shell`](#-shell-) + [`start`](#-start-) + [`stop`](#-stop-) + [`suspend`](#-suspend-) + [`umount`](#-umount-) + [`version`](#-version-) - [Possible new commands](#possible-new-commands) + [`create`](#-create-) + [`set-primary`](#-set-primary-) + [`unset-primary`](#-unset-primary-) * [References](#references) ### General Goal Provide a special VM instance that is readily available to experiment/test/demonstrate. ### Scope * Overall: daemon, clients (CLI & GUI), and possibly installer/snap packaging. * For now: daemon, CLI (not GUI), and possibly installer/snap packaging. ### Baseline Based on the [references below](#references), a summary of current intentions for this feature could be: 1. Current name: `primary` 2. `primary` created automatically (upon installation, current default image) 3. Users may delete the primary instance 4. Users may set which instance is the primary one 5. Commands operate on `primary` by default (i.e. if no target instance is specified) ### Daemon impacts #### Primary instance lifetime In the simplest scenario, the lifetime of the primary instance closely matches multipass’s own *installation-lifetime*. The daemon creates the pet instance ASAP after it first starts and keeps it indefinitely (this accomplishes proposition 2 above, as the daemon is launched when multipass is installed). Uninstalling multipass finally removes `primary` along with any other existing instances (stopping the daemon stops any running VMs and uninstalling removes persistified data). However, a primary instance may also be deleted. Moreover, a different instance may be set as primary (although a maximum of one primary instance is supported at any one time). Creation/destruction and set-primary/unset-primary times are discussed below (not to be confused with start/stop times). In any case, multipass needs to function properly with and without a primary instance. ##### Creation Preconditions may be unfulfilled when the daemon first starts and tries to create the primary instance (e.g. no network to download the image, not enough disk space). *Decision*: What should we do in this case? Options: 1. Ignore and keep going without `primary` 2. Keep going without `primary`, retry periodically *until success* Note: 1 is easier than 2 and pretty much contained in it. So, if we want 2, perhaps it can be done in a separate issue/PR. This is the approach I would favor. *Warning*: IIUC, multipass is already supposed to handle multiple client requests simultaneously (it did respond well to quick smoke tests with concurrent *but orthogonal* requests). However, there could be other concurrency issues when a client is used right away, while `primary` is still launching (see [below](#set-primary)). ###### Feedback to the user Should the setup of `primary` be synchronous with installation, or asynchronous? That is, should the installation wait until `primary` is setup before completing? Current design discussions indicate not, IIUC, but the feedback to the user may be less than ideal in that case. Ideally the user would be notified that an instance was being created/downloaded/launched. They probably want to know that a substantial download is under way and that they have a VM setup. This is especially important if the instance is automatically *started* (see [below](#create-vs-launch)). *Decision*: do we rely only on the tray indicator for this? Is the tray/GUI client meant to be shipped within the same multipass package? What happens in text-only mode? Do we leave the user uninformed until he tries to attach to the instance? Options: 1. info message in installation -> would be easily missed; I think each snap installation message is erased by the one that follows? Can we print a "sticky" message (one that moves output to a new line)? 2. creation process synchronous with installation; e.g. installer outputs creation progress, waiting for it to finish -> would certainly make it noticeable, but delay installation. 3. option during installation ("Do you want to start a primary instance? [Y/n]") -> I guess those should be kept to a minimum(?) Some deb packages have such interactive installation, but I haven’t seen it in a snap... Options 2 and 3 above would effectively make the installer a client of the multipass daemon. Perhaps an indirect one, if it called the CLI client and echoed its output. This has implications on how much new functionality needs to be exposed as interface. In any case, I suggest starting with 1, but together with option 1 in [Create vs Launch](#create-vs-launch) (that is, creating but not starting `primary`. The other options here could be separately implemented, later on, if so decided. Note: any of the options above would expand the scope to the installer level (snap), even if the impact of option 1 would be limited to a single output message. ###### Create vs Launch *Decision*: the instance should be *created* upon installation, but do we also want to start it? Options: 1. No -> A separate create operation would have to be added. Currently, multipassd provides no operation to simply create an instance without starting it. This could be extracted from the current `launch`, which would internally become a combination of `create` and `start` (that is, conceptually, what launching is). A separate *decision* would be whether to expose `create` to clients (so they could offer a `create` command). 2. Yes -> *Feedback* to the user becomes especially important; extra step needed to stop VM if not needed; I would favor option 1. ##### Destruction The user may delete a primary instance with a regular delete request. I suggest it retains the primary status after deleted and until it is purged or the status removed/changed explicitly. All instances are removed at uninstallation at the latest. #### Set primary The daemon needs to extend its interface, to provide a way to set an instance as primary. This can be achieved with a new request, say `set-primary`, with a single existing instance as argument. The operation would move the primary status from any previous instance holding it, *in an atomized step*, to the new intended primary instance (including persistification). By *atomized step* I mean that other threads should not see intermediate states. In other words, concurrent threads serving multiple `set-primary` client requests should be synchronized and execute those operations sequentially. A consistent state, with a single primary instance, needs to prevail in the end. In practice this can be done with a lock around this critical code. *Warning*: I am uneasy about other operations’ current thread safety. For instance, what happens if two clients try to delete and recover an instance concurrently? Looking at the code, this does not seem to be covered. It was probably never noticed because typical usage currently employs a single client. But that will change once we have a tray icon/GUI. Also, concurrent requests will become more likely if `primary` is created asynchronously: the user may start using multipass right after installation, while `primary` is still being created. ##### launch as primary *Decision*: should the `launch` request (and `create`, if we decide to add it) provide a knob to set the new instance as the primary one? Options: 1. Yes -> that would provide the interface to create `primary` with a single request, which would be useful if we opted for the synchronous alternatives in [Feedback to the user](#feedback-to-the-user). The commands would have to consider thread-safety 2. No, only allow setting existing instances as primary -> two steps required to achieve the same thing I think I would favor 1, but perhaps as a later addition. ##### Unset primary *Decision*: do we provide a new "knob" to remove the primary status of an instance? Or can this state only be overwritten? Options: 1. Yes: new request, say `unset-primary` -> one more interface element to support, more functionality to synchronize; one more command to list in help may contribute to obscure essentials 2. No -> the user may feel "uncared for" (he can still achieve the same thing by setting a new instance as primary before deleting it) I think I would favor 2 here. ##### Primary name vs attribute The term `primary` is currently used in both the instance name and to distinguish the status of an instance that receives special treatment. But the possibility to set/unset arbitrary instances as primary introduces a *simetry break*: the status is moved, but the name isn’t. This can lead to confusion, especially when `primary` is no longer *primary*. A simple solution is to use different terms for instance name and status. *Decision*: what terms do we use? Options: 1. Name the primary instance `primary` but start calling something else to the status (e.g. "default") -> but a `primary` instance that was no longer the default could still be confusing 2. Give the instance some other fixed name (e.g. `default`) and continue using "primary" for the status -> analogous problem, although perhaps slightly better 3. Name the instance variably, with a pet name, just like any other instance, still reserving the term "primary" for the status -> makes it clear that primary status cannot be determined from name 4. Rename instances dynamically when setting primary status, so that the current primary instance would always be called `primary` -> would introduce a lot of complication in the way instance records are kept in memory and disk and it would still be confusing to have the name `primary` designate different instances at different times. I favor option 3. Note: this document still uses `primary` to refer to the primary instance in many places #### Default target instance Many multipass commands receive the instances they target as arguments. Commands that support multiple instances also provide the flag `--all` to target all instances. An instance argument is required unless the flag `--all` is present (e.g. `multipass start --all`). [Baseline](#baseline) proposition 5 means that commands with neither instance arguments nor the `--all` flag would target the primary instance (if available). So something like `multipass stop` would mean the same as `multipass stop primary`. However, client requests with an empty instance field are currently interpreted by the daemon to mean that all instances are to be targeted. So, to accomplish this default-instance feature, one of two things would need to happen: 1. the logic of checking for a default target and selecting it would have to be placed in the client 2. the client/server communication protocol would have to be changed in all such requests From these options, I favor the second, but I wonder... *Decision*: is this default instance behavior really what we want? I am not sure the pros compensate the cons, especially when considering that: * if only `primary` exists, *auto-complete already selects it* with a simple tab key press * if other instances exist, muscle memory could cause people to target `primary` by mistake * `primary` will often *not be an appropriate default*. For instance, it would not be appropriate to say `multipass start` if `primary` was trashed; or `multipass purge` unless `primary` was deleted; or `multipass info` if no primary was set -> A default that does not account for that makes it easy to try wrong things. And only the daemon is in the proper place to discern that. Unless the client made preliminary requests... Again, autocomplete already deals with this sort of thing * if we removed this default targeting functionality, the only thing distinguishing a `primary` instance would be that it was created automatically! *Primary status* would otherwise be irrelevant, so it would make no sense to support moving it. That would greatly simplify this feature: *much of the complexity discussed here would disappear* * I think the "default instance" functionality has no counterpart in the GUI client, so the effect of setting an instance as primary may be unclear to someone using only the GUI In my opinion, we should strive for simplicity here. Are the additional complications worth the occasional typing economy? I am not trying to argue for something like Apple’s single button approach, which I don’t really like (everyone has edge cases once in a while). But a balance should still be struck. I find that removing the "default instance" feature could actually contribute more to a "slick" multipass usage experience than the other way around. Detail: if the user wants to target `primary` in a command that also targets other instances, they have to mention it explicitly. It is perhaps worth highlighting that a command may have 2 more arguments than another and yet only target one additional instance. This is a potential pitfall for both developers and users: it is easy to mistake the number of arguments with the number of targets (e.g. imagine a script’s maintainer trying to add an additional target to an exiting command-line variable). Note: this document still deals with the current state of affairs: a Pet Environment feature that includes the default instance. #### Persistifying The instance with primary status has to be persistified. Since only one is allowed at any point, this could be achieved with a separate field, which could either be empty or specify a single instance. The field should be saved whenever the corresponding status changes in memory (i.e. set/unset primary, launch, etc.). #### Canceling a start-up Canceling an instance creation procedure is mentioned a few times in multipass-design. While this could be supported by multipassd, this is not currently the case. SIGINT on CLI only aborts the client, but the daemon keeps handling the request. Implementing this would require actions to be transactional, so they could be rolled back, lest we get into inconsistent intermediate states. It would be a big feature on its own. ### CLI client impacts #### Existing commands ##### `--help` The command itself is unaffected. The documentation it relies upon needs to change in the following ways: * If we create [new commands](#Possible-new-commands), they need to be documented * If we want to target `primary` by default, change the documentation of all affected commands to explain this, mentioning that it only works if the instance exists (and that other requirements still apply) ##### `--verbose` No direct impact, just accompany new functionality with appropriate logging. ##### `copy-files` If we decide to target `primary` by default in other commands, we may choose to abbreviate `primary:` as `:` here. Omitting the instance still requires a leading colon, to distinguish which end of the copy is inside the instance (`multipass copy-files /a/path /another/path` would be ambiguous). The command would fail if we defaulted to `primary` and its state was not `RUNNING`. ##### `delete` A default instance would be applicable here if so decided. *Decision*: would we allow a default also with `--purge`? Options: 1. Yes -> dangerous, too punishing on stray-enter mistake 2. No -> another special case, another asymmetry The command would be a *no-op* if we defaulted to `primary` and its state was `DELETED`. ##### `exec` A default instance would be applicable here if so decided. If so, then the argument `--` becomes mandatory when no instance argument is provided, to distinguish where the command begins. In other words, the first free argument would still be an instance name when before `--`. The command would fail if we defaulted to `primary` and its state was not `RUNNING`. ##### `find` No impact I can think of. ##### `help` See [`--help`](#-help) ##### `info` The output should clearly indicate when an instance was tagged as primary. The primary instance should also be listed first, when part of the output. A default instance here could confuse expectations. In particular, a distracted user may expect `multipass info` to encompass all instances. Note: today, output order simply follows argument order. ##### `launch` If we decide to support launching an instance as primary (see [launch as primary](#launch-as-primary)) this command would accept an additional flag (e.g. `--set-primary`). Note: this command receives a free argument specifying an image. That argument already has a default, but here it designates the image rather than an instance. This means that there is a small asymmetry relatively to the other commands, which may present confusing affordance clues. The asymmetry is not introduced by the default-instance feature on this occasion – it is already there. But its impact may be aggravated. In particular, the unfamiliar user may confuse `launch` with `start` and expect `multipass launch` to mean `multipass start [primary]`, although they are quite different. ##### `list` The primary instance should be listed in the first place. *Decision* should the output somehow highlight the primary instance? Options: 1. No -> ...but it would be nice :) 2. Yes, with a dedicated column -> but the output table is already quite wide... 3. Yes, with a single character mark (perhaps '*'?) -> would have to be documented; where to put it? beginning or end of name? I would favor option 3, end of name. ##### `mount` As in [`copy-files`](#copy-files), if we decide to target `primary` by default in other commands, we may choose to abbreviate `primary:` as `:` here. Unlike in the former command, mount endpoints are distinguished by position, so the leading colon would not be strictly required. But I suggest maintaining it, for consistency and clarity. The command would fail if we defaulted to `primary` and its state was not `RUNNING`. ##### `purge` A default instance would be applicable here if so decided. *Decision*: do we allow a default here in particular? Options: 1. Yes -> dangerous, too punishing on stray-enter mistake 2. No -> another special case, another asymmetry The command would fail if we defaulted to `primary` and its state was not `DELETED`. ##### `recover` A default instance would be applicable here if so decided. The command would be a `no-op` if we defaulted to `primary` and its state was not `DELETED`. ##### `restart` A default instance would be applicable here if so decided. The command would fail if we defaulted to `primary` and its state was not `RUNNING` (actually depends on the outcome of [this issue](https://github.com/CanonicalLtd/multipass/issues/545). ##### `shell` A default instance would be applicable here if so decided. The command would fail if we defaulted to `primary` and its state was not `RUNNING`. ##### `start` A default instance would be applicable here if so decided. The command would be a `no-op` if we defaulted to `primary` and its state was `RUNNING`. The command would fail if we defaulted to `primary` and its state was `DELETED`. ##### `stop` A default instance would be applicable here if so decided. The command would be a `no-op` if we defaulted to `primary` and its state was either `STOPPED` or `SUSPENDED`. The command would fail if we defaulted to `primary` and its state was `DELETED`. ##### `suspend` A default instance would be applicable here if so decided. The command would be a `no-op` if we defaulted to `primary` and its state was `SUSPENDED` or `STOPPED`. The command would fail if we defaulted to `primary` and its state was `DELETED`. ##### `umount` As in [`copy-files`](#copy-files), if we decide to target `primary` by default in other commands, we may choose to abbreviate `primary:` as `:` here. In this case, and since we may already omit paths to unmount all mounts in an instance, we should also support `multipass umount` to unmount all mount points in the primary instance. Omitting the instance still requires the leading colon, for clarity and consistency with `copy-files`, and to provide a quick way to distinguish instance names from paths. The command would be a `no-op` if we defaulted to `primary` and there were no mount points in that instance. The command would fail if we defaulted to `primary` and its state was `DELETED` (unless [this issue](https://github.com/CanonicalLtd/multipass/issues/560) turns out to be valid). ##### `version` No impact. #### Possible new commands ##### `create` If it was added, this would have the same interface as `launch`, except it would not start the created instance. See the [Create vs Launch](#create-vs-launch) discussion above. ##### `set-primary` This receives a single free argument, identifying an existing instance to set s primary. Whether we require that instance not to be deleted should match what we do in [`umount`](#umount). See the section [set primary](#set-primary) above. ##### `unset-primary` If this was added, it would not receive any arguments and it would remove the primary status from the instance holding it, if any (`no-op` otherwise). Whether we require that instance not to be deleted should match what we do in [`umount`](#umount). ### References * https://github.com/CanonicalLtd/multipass-design/issues/18 * https://github.com/CanonicalLtd/multipass-design/issues/19 * https://github.com/CanonicalLtd/multipass-design/issues/20 * https://github.com/CanonicalLtd/multipass-design/issues/21 * https://github.com/CanonicalLtd/multipass-design/issues/22 * https://docs.google.com/document/d/1NW53qrqJDCfjOgpcwoptsO9-L1sO-IInD-dIKEkpGJc/edit#heading=h.jgqsclo3jmsi