De acordo com a documentação as rotas padrões tem que redirecionar pro index https://angular.io/guide/deployment#routed-apps-must-fallback-to-indexhtml
reescreva o .htaccess
RewriteEngine On
# If an existing asset or directory is requested go to it as it is
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR]
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d
RewriteRule ^ - [L]
# If the requested resource doesn't exist, use index.html
RewriteRule ^ /index.html
para configuração especifica do apache: https://ngmilk.rocks/2015/03/09/angularjs-html5-mode-or-pretty-urls-on-apache-using-htaccess/
OUTRA SOLUÇÃO
Ao invés de atualizar seu servidor para servir o index.html, basta mudar para a abordagem HashBand.
Importar no app.module.ts:
import { HashLocationStrategy, LocationStrategy } from '@angular/common';
E adicionar no NgModule:
{provide: LocationStrategy, useClass: HashLocationStrategy}