Docker Reverse Proxy to React App with React Router - reactjs

Question

I've been hitting my head on the keyboard for 3 days trying to figure out why I can't get this to work.
Right now I am running two nginx servers on my machine. Both are running in docker containers.
One nginx is a reverse proxy. The other nginx is the actual one serving the static files built by 'yarn build'.
I currently get a 404 error when trying to access any of the files out side of the root file.
So example.com works just fine. But when I go to example.com/test (a react route) I get a 404 error.
So, here is my reverse proxy confiruation.
server {
listen 80;
server_name example.com www.example.com;
location / {
proxy_pass http://192.168.1.29:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
location ^~ /.well-known/acme-challenge/ {
root /var/www/html;
}
}
Here is my actual example.com confiration to serve the static files (yes it's port 80 because it's running on port 80 inside the docker container. The exposed docker container port is 8080:
server {
listen 80;
listen [::]:80;
server_name localhost;
root /var/www/html;
try_files $uri $uri/ index.html;
index index.html;
location / {
}
}