Using SSH keys is the best way to make sure others can't get into your VPS. It makes it so the only people that can ever connect are the ones with a SSH key file.
To connect to your Linux server using an SSH key follow these steps.
Open up putty and put in your connection information. The hostname/ip of your server and the correct port
Now click on the plus sign (+) next to the "SSH" portion of the putty settings on the left as seen in the image below. This will expand those settings and show those settings.
After you have done that click on the "Auth" and you will see a white box where you can select a file. Simply select the private key file for your SSH key.
Last you might want to save your settings so you don't need to add the key file each time. Click the "Session" section of the settings on the left and it will take you back to the default that you are used to. Type a name that you want to save as in the white box under "Saved Sessions" and then hit the save butto to the right of it. You will now notice that it saves all the current settings of putty, the location of the keyfile and the hostname and port you have above. In the future all you need to do is click the saved session and it will load everything up for you.