"The program can't start because sqlite3.dll is missing from your computer. Try reinstalling the program to fix this problem."
After quite a lot of googling I found that the problem was that the sqlite3 gem
(installed with the
bundle installin step 3.3 I believe) didn't include windows DLL. This is probably because "most Ruby developers use Linux".
The solution is to download the DLL from the official SQLite website http://www.sqlite.org/download.html. You want the "This ZIP archive contains a DLL for the SQLite library" file. At the time of writing it is the following file: http://www.sqlite.org/sqlite-dll-win32-x86-3070600.zip
Put the files from the ZIP file into your ruby/bin directory. Close and open a new console window and continue with the Ruby on Rails guide.