Token expiration issue using oidc-client? - identityserver4

Question

I have a site which is using IS4 and the front end is Angular 7. I have the oidc-client library to handle all user authentication etc and everything works fine. This is the first time I'm seeing this issue though. I can log in and interact just fine, with my permissions, role stuff and it's all good. Sometimes I get a strange error after some inactivity so when I try to perform a secured operation I get an error straightaway without even hitting my API. If I refresh the page and perform the action, it works fine.
I have implemented the silent renew callback in my component just like this:
#Component({
selector: 'app-silent-renew-callback',
templateUrl: 'silent-renew-callback.component.html'
})
export class SilentRenewCallbackComponent implements OnInit {
constructor(private authService: AuthService) { }
ngOnInit() {
this.authService.signingSilentCallback();
}
}
which then goes off to the authService code:
signingSilentCallback(): Promise<void> {
return this.manager.signinSilentCallback()
.catch(e => {
console.log(e);
});
}
My auth service event configuration looks like this:
#Injectable()
export class AuthService {
private manager: UserManager;
private user: User = null;
constructor() {
if (!environment.production) {
Log.logger = console;
}
this.manager = new UserManager(getClientSettings());
this.manager.getUser()
.then(user => {
this.user = user;
});
this.manager.events.addUserSignedOut(() => {
this.signOut();
});
this.manager.events.addAccessTokenExpired(() => {
this.signOut();
});
}
}
export function getClientSettings(): UserManagerSettings {
return {
authority: environment.authorityUrl,
client_id: 'my_client_id',
redirect_uri: `${environment.baseUrl}/auth-callback`,
post_logout_redirect_uri: environment.baseUrl,
response_type: 'id_token token',
scope: 'openid profile my_api',
filterProtocolClaims: true,
loadUserInfo: true,
automaticSilentRenew: true,
silent_redirect_uri: `${environment.baseUrl}/silent-renew-callback`,
};
}
The thing is that reading some posts it looks like there should be an event hooked up when you renew the user:
this.manager.events.addUserLoaded(_ => {
this.manager.getUser()
.then(user => {
this.user = user;
});
});
My question is whether that's mandatory to be added when you use the silent renew or not and also, in case that is not needed, if that rings the bell of any issues you guys have come across this before.
Thanks