Could not start backup:Request failed with status code 400 Google Cloud - google-app-engine

Question

I'm trying to create a backup system for Firestore.
I followed every step of this guide and when I tried to deploy the code, it returned Request failed with status code 400
PROJECT-ID#appspot.gserviceaccount.com permissions: Cloud Datastore Import Export Admin,
Editor,
Storage Admin
This is the code of app.js
'use strict';
const axios = require('axios');
const dateformat = require('dateformat');
const { google } = require('googleapis');
const express = require('express');
const util = require('util')
const request = require('request');
const admin = require('firebase-admin');
const {Storage} = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
admin.initializeApp({
credential: admin.credential.applicationDefault()
});
const db = admin.firestore();
const googleMapsClient = require('#google/maps').createClient({
key: 'AIza*****',
Promise: Promise
});
const app = express();
// Trigger a backup
app.get('/cloud-firestore-export', async (req, res) => {
const auth = await google.auth.getClient({
scopes: ['https://www.googleapis.com/auth/datastore'],
});
const accessTokenResponse = await auth.getAccessToken();
const accessToken = accessTokenResponse.token;
const headers = {
'Content-Type': 'application/json',
Authorization: 'Bearer ' + accessToken,
};
const { outputUriPrefix } = req.query;
if (!outputUriPrefix) {
res.status(500).send('outputUriPrefix required');
} else if (outputUriPrefix && outputUriPrefix.indexOf('gs://') !== 0) {
res.status(500).send('Malformed outputUriPrefix: ${outputUriPrefix}');
}
// Construct a backup path folder based on the timestamp
const timestamp = dateformat(Date.now(), 'yyyy-mm-dd-HH-MM-ss');
let path = outputUriPrefix;
if (path.endsWith('/')) {
path += timestamp;
} else {
path += '/' + timestamp;
}
const body = {
outputUriPrefix: path,
};
// If specified, mark specific collections for backup
const { collections } = req.query;
if (collections) {
body.collectionIds = collections.split(',');
}
const projectId = process.env.GOOGLE_CLOUD_PROJECT;
const url = 'https://firestore.googleapis.com/v1beta1/projects/' + projectId + '/databases/(default):exportDocuments';
try {
const response = await axios.post(url, body, { headers });
res
.status(200)
.send(response.data)
.end();
} catch (e) {
if (e.response) {
console.warn(e.response.data);
}
res
.status(500)
.send('Could not start backup:' + e.message)
.end();
}
});
°°°°
// Start the server
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log('App listening on port ${PORT}');
console.log('Press Ctrl+C to quit.');
});
I have another function that is listening to '/'. Is possible that this can cause the problem?
package.json:
{
"name": "solution-scheduled-backups",
"version": "1.0.0",
"description": "Scheduled Cloud Firestore backups via AppEngine cron",
"main": "app.js",
"engines": {
"node": "10.x.x"
},
"scripts": {
"deploy": "gcloud app deploy --quiet app.yaml cron.yaml",
"start": "node app.js"
},
"author": "Google, Inc.",
"license": "Apache-2.0",
"dependencies": {
"#google-cloud/storage": "^3.2.1",
"#google/maps": "^0.5.5",
"axios": "^0.19.0",
"dateformat": "^3.0.3",
"express": "^4.17.1",
"firebase-admin": "^8.4.0",
"googleapis": "^42.0.0",
"request": "^2.88.0"
},
"devDependencies": {
"prettier": "^1.18.2"
}
}
I also look inside Cron log and there is nothing related to the error. It only returns 500 error

Answer 1

The main problem is the bucket location
When you create a backup bucket you must use Multi-Region or you will receive a deny from the server.
I think this is a bug of Google Cloud
Solution
Delete the bucket and create a new one with a Multi-Regional location
The error with Single Location:
'Bucket backup-bucket is in location EUR4. This project can only operate on buckets
spanning location europe-north1 or europe-west1 or eu or europe-west2 or
europe-west3 or europe-west4 or europe-west5 or europe-west6.',

Answer 2

So i've been replicating your issue and i've found the solution.
I kept getting the same error as you and I finally managed to figure it out. If you look into GAE logs you can see the error saying that 'Project \' "YOUR PROJECT" \' is not a Cloud Firestore enabled project.'.
This worked for me:
Make a new project.
Go to API Library in GCP and enable Firestore Installation API.
Go to Firebase and link your GCP project to a firebase project.
Go to database and create Firestore Database.
Follow the repo with the permissions and the deployment to app engine.
Test the cron and it will be successful.
If you ever had DataStore enabled in your actual project you will not be able to do a Firestore instance at step 3.
This will create you the needed buckets with the .appspot.com format, that you have give permissions to.
Go to GAE and create your cron.yaml, app.js and everything else you need. I used this repo for tests.
In the readme.md of the repo you have the exact commands that you have to do in order to give permissions to your service account.
Remember to change the bucket as told in cron.yaml.
Follow the steps mentioned in the repo as they are pretty well done.
Let me know if it worked for you!