Troubleshooting the Bluesky PDS (Personal Data Server)

Bluesky, one of the new Social Network contenders, recently made its first steps towards becoming properly federated with the launch of a self hosted option for the Personal Data Server. In other words, if you create an account on the PDS, all of your posts, media etc on Bluesky will ultimately be hosted on the PDS meaning that you, or another party you trust, hold the keys to your data. Migration of an existing account off bluesky servers and onto a PDS is possible, but currenly filled with so many disclaimers it’s probably not worth looking at in any serious manner yet.

It’s important to note that Bluesky themselves say that the PDS is not yet stable, and definitely not ready for prime time. That said, I set one up for a test account and from being in the Discord server for PDS admins (link on the main PDS github) it seems like there’s some common pain points if you’re doing something more complex that the automated installer can’t handle. Thus…. my troubleshooting guide.

I’m assuming you’ve already run through the setup guide on the repo and somethings gone wrong.

I suggest running through each section in turn, making sure each test works. For each domain I’ll be using example.com as the PDS domain. Substitute your PDSes own domain. Example responses are cribbed from my live PDS, slightly adjusted to make it clear which parts will be different.

HTTPS Connectivity to XRPC

The PDS requires that the following traffic is forwarded to it. These calls can be tried in your web browser of choice.

  1. Call https://pds.example.com/xrpc/_health

This call should return some simple JSON showing the version of the PDS code you are running, e.g. {"version":"0.4.12"}

If this call fails with an HTTPS error, then you need to double check that your TLS Certificate is valid your your PDS domain such as example.com.

If this call fails with a 503 error ro a similar 5xx error, then it is likely that your reverse proxy isn’t able to talk to your PDS application / container. Double check your reverse proxy configuration for how it’s talking to the PDS. For example it should be sending traffic to the PDS on port 3000.

If you get a 404 error, then check that your reverse proxy is correctly forwarding all /xrpc path traffic to the PDS.

Websocket Connectivity

The PDS requires that websocket connections are correctly Upgraded and forwarded by your reverse proxy. Not every reverse proxy handles this by default, and this seems to be a common pitfall for people going off the ‘golden path’ that the DPS documentation describes.

  1. Get yourself a websocket client. EG in a browser extnesion (I use https://addons.mozilla.org/en-US/firefox/addon/simple-websocket-client/), or a CLI client if you prefer.
  2. Issue a websocket call to wss://example.com/xrpc/com.atproto.sync.subscribeRepos this should result in an open websocket connection.

If you enocunter problems here you need to look up how to correctly handle websockets with your Reverse Proxy, e.g. using the information at http://nginx.org/en/docs/http/websocket.html for NGINX.

Request Federation at this point

At this point we’ve checked as much as we can config wise without an account on the PDS to test with. I would really recommend subimtting your federation request at this point and waiting for it to be approved, if you haven’t already. As the rest of the troubleshooting requires an account, and not being federated yet can lead to spurious problems as the bsky network will refuse to talk to your PDS properly.

Request a crawl at this point

Now that you are federated, you should request a crawl, especially if you were federated prior to finding this guide and you have fixed a problem.

  1. Use the pdsadmin request-crawl command. Or take a look at the commands source and manually craft the curl request to your instance

HTTPS Connectivity to your PDSs subdomains

To support accounts being created using the subdomain of your PDS – which is how all the bsky apps require you sto sign up right now, it is important that you are correctly able to handle requests to subdomains. We’ll be using a freshly created account here, to ensure that the actual contents should be valid.

  1. Create an account on your pds. eg with pdsadmin account create or, as above, look at the commands source and manually craft the curl request
  2. Visit https://bsky-debug.app/handle and enter youraccount.example.com and check that it shows as being valid by the HTTP method. You can ignore the DNS method for this test.

If it shows a TLS error, then ensure that your reverse proxy has a TLS Certificate configured for *.example.com to be used on all subdomains of example.com; or that it can get certificates for subdomains of example.com on demand (such as caddy does)

If you are getting a 404 error then ensure the following is setup

  • traffic for *.example.com is being forwarded to the PDS
  • traffic for .well-known/atproto-did is being forwarded to the PDS

If you are still having problems try loading https://youraccount.example.com/.well-known/atproto-did in your web browser yourself. You should receive back something like did:plc:abc123def456ghi789

One final test to close the loop on the account

One final test would be to call https://example.com/xrpc/com.atproto.repo.describeRepo?repo=did:plc:abc123def456ghi789 in your web browser, using the did that matches the account you made in the last step. This should return a JSON document containing a bunch of information about your did, including that the handle is youraccount.example.com.

That’s all folks

That’s all I’ve got. By my understanding if everything I’ve described here is working then your PDS is properly working and you should be able to do anything you want through the bsky clients, including changing your handle. I’ll be keeping an eye on the PDS Admin discord, and may update this as I see other pain points that people going off the golden path hit, and ways to trobuleshoot them.

Installing remux, KOReader on the ReMarkable 2

IMPORTANT UPDATE:

The ecosystem for third-party ReMarkable software has moved on a lot since I wrote this post originally. I’m working on a new guide that will hopefully be a lot more up-to-date. Various packages have been replaced and the toltec install method has changed.

I’d originally intended this to mostly be notes for myself to document what I’d did, but apparently various folks found it useful, so a new version is available here

The Original Post

I pre-ordered the ReMarkable 2 a long time ago, and now that I’ve had it for a few months, and have been very satisfied with it I took a look at the state of the hacking community that sealed the deal for me in the first place – ReMarkable provide out of the box SSH access.

After watching and monitoring several github issues for a while about reMarkable 2 support they eventually reached the point where I decided to give it a try, even if things weren’t going to work perfectly….. and it went totally smoothly!

Important caveats: A lot of this is still beta quality at best. This worked when I wrote it.

Here’s what I did

  1. Ensure you are up to date with reMarkable software
    – Menu > Settings > Software Version > Check for Updates (As of this post the latest is 2.5.0.27)
  2. Obtain the ReMarkables IP Address and SSH Password
    – Menu > Settings > Help > Copyrights and Licenses > General Information > Look at the bottom
    – WRITE THE SSH PASSWORD DOWN SOMEWHERE SAFE. In case you can’t get back to this screen should something go wrong.
  3. SSH in.
    ssh root@<ip> and provide the password
  4. Install Toltec package repository
    – Run the three commands in the README
  5. Install rm2fb – A shim for the ReMarkable display
    opkg install rm2fb
  6. Install remux – a GUI launcher to switch between apps
    opkg install remux
  7. Enable remux
    systemctl enable --now remux
  8. Test that remux is working
    – Swipe up from the bottom of the ReMarkable 2. You should see a menu appear with one entry in it.
  9. Install koreader
    opkg install koreader
  10. Configure remux to know about koreader
    – Create a file /home/root/.config/draft/02-koreader with the following contents
    name=koreader
    desc=Ebook reader
    call=/opt/koreader/koreader.sh
    term=;
  11. Test that koreader is working
    – Swipe up from the bottom of the ReMarkable 2 and select koreader from the list.
    – For the app, use the ReMarkable pen to swipe down from the top to access the KOReader menu.
    – Note you can’t read your ReMarkable documents in KOReader as even ePubs appear to be converted in to an incompatible format (inside /home/root/.local/share/remarkable/xochitl for the curious)
  12. Congratulations, you are done!

Note that when a ReMarkable system update occurs you will likely need to re-run the systemctl enable --now remux command. I think this should be the only command needed to re-run, but I don’t have a system update to install to validate that.

Addendum

It’s been mentioned to me privately that reMarkable software updates occasionally reset teh SSH password. For this reason it would be advisable to setup an SSH Key for the device, as there is apparently no known instance of the authorized_keys file being reset. Instructions to do this are out of scope, but are easily googleable as it works the same as any other device.

A final point was mentioned in the comments of this post which is that it might also be necessary to re-run /home/root/entware-reenable after a system update, just like you have to reru nthe remux enablement command.