First, ssh into your instance and edit your code-server config file to disable password authentication.

# Replaces "auth: password" with "auth: none" in the code-server config.
sed -i.bak 's/auth: password/auth: none/' ~/.config/code-server/config.yaml
Restart code-server with (assuming you followed the guide):

sudo systemctl restart code-server@$USER
Now forward local port 8080 to 127.0.0.1:8080 on the remote instance by running the following command on your local machine.

Recommended reading: https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding.

# -N disables executing a remote shell
ssh -N -L 8080:127.0.0.1:8080 [user]@