According font-awesome docs here, after install package

yarn add --dev @fortawesome/fontawesome-free

or

npm install --save-dev @fortawesome/fontawesome-free

Require font-awesome into your config file (in my case and default Symfony 4 location) assets/js/app.js:

require('@fortawesome/fontawesome-free/css/all.min.css');
require('@fortawesome/fontawesome-free/js/all.js');

Compile again yarn encore dev and icons should appear.