Plan for user testing of sq

By Lars | August 4, 2022

I will do some informal user testing of sq. In short, I will watch volunteers use sq to achieve specific tasks that I give them.

The goal of this is to find out pain points when using sq: what is easy and straightforward; what is difficult to understand; what is difficult to do. The testing will cover the sq command line tool and its built-in help, but not any other manuals or materials. The outcome I hope for is a list of proposed improvements to sq. The volunteers will not be judged or graded.

The form of the user testing is a set of tasks given to the volunteer to perform, while I observe via a video conferencing system.

The tasks to perform are:

  • Log into the virtual machine.
  • Generate a new key.
  • Share the certificate for the new key with me.
  • Get my certificate.
  • Certify my key and share the result with me.
  • Encrypt a message using my certificate, and send the encrypted message to me
  • Receive a response from me. Verify and decrypt the response.

Test setup:

  • I will provide a virtual machine with a specific version of sq installed. This is to avoid “installing sq” being part of the test, and also to make sure the version of sq being used works as expected.
  • The volunteer will access the virtual machine over SSH.
  • The volunteer will share their screen, or at least their terminal window where they access the virtual machine, over the video conferencing system being used.
  • I will destroy the virtual machine after the test.
  • Instead of email the volunteer and I will exchange files via a mutually writable directory on the virtual machine.

I will take notes and publish a blog post about what I learn.

Note

This work is supported by a grant from the NLnet foundation from the NGI Assure fund, which is financially supported by the European Council.