Hi i am trying to create a simple node app on google app standard app engine using this terraform code. This code used to work before but today i was trying to restart the whole project and re-deploy everything again and i see that i am getting an error.
compute_engine.tf
resource "google_app_engine_standard_app_version" "nodetest" {
version_id = "v1"
service = "mainApp"
runtime = "nodejs10"
instance_class = "B1"
basic_scaling {
max_instances = 1
}
entrypoint {
shell = "node test.js"
}
deployment {
files {
name = google_storage_bucket_object.object.name
source_url = "https://storage.googleapis.com/${google_storage_bucket.bucket.name}/${google_storage_bucket_object.object.name}"
}
}
delete_service_on_destroy = true
depends_on = [
google_project_service.appengine_api
]
}
resource "google_storage_bucket" "bucket" {
project = var.project_id
name = var.bucket_name
location = var.region
}
resource "google_storage_bucket_object" "object" {
name = "test.js"
bucket = google_storage_bucket.bucket.name
source = "test.js"
}
My test.js is located in the same directory as where tf is located.
test.js
const http = require('http');
const hostname = '127.0.0.1';
const port = 3000;
const server = http.createServer((req, res) => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('Hello World');
});
server.listen(port, hostname, () => {
console.log(`Server running at http://${hostname}:${port}/`);
});
I see that the files have already been deployed correctly
And the error i am getting
I tried changing the url from
"https://storage.googleapis.com/${google_storage_bucket.bucket.name}/${google_storage_bucket_object.object.name}"
To
"https://storage.cloud.com/${google_storage_bucket.bucket.name}/${google_storage_bucket_object.object.name}"
Try changing the shell = "node test.js" to shell = "node ./test.js"
Also i did take a look at GitHub Issue 4974 but is doesnt solve my problem. I did notice that when i try to terraform apply the error pretty much pop up quite fast so it seem that is stuck on a very first validation error.
Does the user that runs compute_engine.tf has "appengine.applications.create" and deploy permissions?
Also check if you set project and region in your google provider.
Related
I'm trying to load the contents of files in a specific folder as env vars accessible at process.env.SomethingHere at runtime not build time as those files don't exist at build time yet (if process.env is the right place to load those secrets, more about that in the end). I managed to write the code to read the files and create a key-value object/dictionary from them, but not sure where is the best spot in the app lifecycle to add this so it's available for both server-side middleware and client-side frontend
function readFiles(dir) {
var vars = {};
if(!fs.existsSync(dir))
{
console.log("failed to load files in directory: " + dir);
return vars;
}
// read directory
let fileNames = fs.readdirSync(dir);
if (!fileNames) {
console.log("failed to load files in directory: " + dir);
return vars;
}
fileNames.forEach((filename, index) => {
// get current file name
const name = path.parse(filename).name;
// get current file path
const filepath = path.resolve(dir, filename);
// get information about the file
let stat = fs.statSync(filepath);
if (!stat) {
console.log("failed to load file stats: " + name);
return;
}
// check if the current path is a file or a folder
const isFile = stat.isFile();
// exclude folders
if (isFile) {
let data = fs.readFileSync(filepath);
if (!data) {
console.log("failed to load file: " + name);
return;
}
let content = data.toString('utf8', 0, data.length);
// callback, do something with the file
vars[name] = content;
}
});
return vars;
}
Some background:
I'm using Azure keyvault for storing secrets and have a Kubernetes cluster with a frontend and a backend pod.
In the backend pod I was able to mount the secrets using CSI driver fetch the values at app startup and load that into the python os.environ. I'm trying to achieve the same for the frontend pod, the mounting is successful, but I haven't been able to place my code in the right spot to load the secrets into process.env if this is even the right place to add them.
ChatGPT solved the issue eventually, here is the answer:
In a Next.js application, the server-side entry point is the server.js file located in the root directory of the project. By default, this file exports the createServer function, which is used to create the Express server that handles server-side rendering of the application.
You can add your code to load the environment variables in this file by importing the readFiles function and using the Object.assign() method to merge the key-value object with the existing process.env object, like this:
env.js
function readFiles(dir) {
// existing code here
}
module.exports = readFiles;
server.js in the root directory of the project
const readFiles = require('./env');
const dir = './dir_name'
const envVars = readFiles(dir);
Object.assign(process.env, envVars);
const dev = process.env.NODE_ENV !== 'production'
const hostname = 'localhost'
const port = 3000
const {createServer} = require('next');
const app = next({ dev, hostname, port })
const handle = app.getRequestHandler();
app.prepare().then(() => {
createServer(handle).listen(port, (err) => {
if (err) throw err;
console.log(`> Ready on http://${hostname}:${port}`)
});
});
I am building a newsletter sign-up form that uses .netlify-lambda to send my form submission to Mailchimp. I installed http-proxy-middleware to help the front end find the netlify-lambda folder. After writing the proxy setup code below my React start script stopped working. It appears the proxy setup below is interfering with localhost:3000.
My proxy setup looks like this
const proxy = require('http-proxy-middleware');
module.exports = function(app) {
console.log('Using proxy...')
app.use(proxy('/.netlify/functions/', {
target: 'http://localhost:9000/',
"pathRewrite": {
"^\\.netlify/functions": ""
}
}));
};
If the target is localhost:9000 why is it interfering with localhost:3000?
When I start my Lambda server it says: Lambda server is listening on 9000.
I am also getting this error when trying to compile my client app.
crbug/1173575, non-JS module files deprecated
Short answer (for #lachnroll and anyone who might be encountering the same problem):
Please use const { createProxyMiddleware } = require("http-proxy-middleware") and app.use(createProxyMiddleware('/.netlify/functions/' ...)...) , instead of using const proxy = require('http-proxy-middleware'); and app.use(proxy("/.netlify/functions/" ...)...) , it should work.
Long one:
I've come across the same "can't be reached" thing in a React project when using http-proxy-middleware(2.0.3), until I changed const proxy = require('http-proxy-middleware'); and proxy("/.netlify/functions/" ...) to const { createProxyMiddleware } = require("http-proxy-middleware"); and app.use(createProxyMiddleware('/.netlify/functions/' ...)...) , I think the proxy has been removed, see: https://github.com/chimurai/http-proxy-middleware#readme
I'm using workbox-webpack-plugin to generate a service worker for me and I'm using copy-webpack-plugin move generated service worker files to the same directory as main.js. My next js config file goes like this:-
module.exports = {
webpack: (config, {isServer, buildId, dev, ...rest}) => {
if (dev) {
const devSwSrc = join(__dirname, "register-sw.js");
config.plugins.push(new CopyWebpackPlugin([devSwSrc]));
config.plugins.push(new GenerateSW({ ...defaultGenerateOpts }));
// Register SW
const originalEntry = config.entry;
config.entry = async () => {
const entries = await originalEntry();
const swCompiledPath = join(__dirname, 'register-sw-compiled.js');
if (entries['main.js'] && !entries['main.js'].includes(swCompiledPath)) {
let content = await readFile(require.resolve('./register-sw.js'), 'utf8');
await writeFile(swCompiledPath, content, 'utf8');
entries['main.js'].unshift(swCompiledPath);
}
return entries;
};
}
I'm trying to copying my service worker to the same dir as main.js which is chunk/static so that when it's fetched it should not return any error. But instead, I'm getting this error.
TypeError: Failed to register a ServiceWorker for scope ('[http://localhost:3000/](http://localhost:3000/)') with the script ('[http://localhost:3000/service-worker.js](http://localhost:3000/service-worker.js)'): A bad HTTP response code (404) was received when fetching the script.
I know this error is because it's not getting served from the same dir as main.js and I need to make some changes in copy-webpack-plugin in order to achieve that. Also I'm trying to avoid custom server.js file to server routes like /service-worker
Any help would be really appreciated. Thanks in advance
I'm trying to create a backup system for Firestore.
I followed every step of this guide and when I tried to deploy the code, it returned Request failed with status code 400
PROJECT-ID#appspot.gserviceaccount.com permissions: Cloud Datastore Import Export Admin,
Editor,
Storage Admin
This is the code of app.js
'use strict';
const axios = require('axios');
const dateformat = require('dateformat');
const { google } = require('googleapis');
const express = require('express');
const util = require('util')
const request = require('request');
const admin = require('firebase-admin');
const {Storage} = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
admin.initializeApp({
credential: admin.credential.applicationDefault()
});
const db = admin.firestore();
const googleMapsClient = require('#google/maps').createClient({
key: 'AIza*****',
Promise: Promise
});
const app = express();
// Trigger a backup
app.get('/cloud-firestore-export', async (req, res) => {
const auth = await google.auth.getClient({
scopes: ['https://www.googleapis.com/auth/datastore'],
});
const accessTokenResponse = await auth.getAccessToken();
const accessToken = accessTokenResponse.token;
const headers = {
'Content-Type': 'application/json',
Authorization: 'Bearer ' + accessToken,
};
const { outputUriPrefix } = req.query;
if (!outputUriPrefix) {
res.status(500).send('outputUriPrefix required');
} else if (outputUriPrefix && outputUriPrefix.indexOf('gs://') !== 0) {
res.status(500).send('Malformed outputUriPrefix: ${outputUriPrefix}');
}
// Construct a backup path folder based on the timestamp
const timestamp = dateformat(Date.now(), 'yyyy-mm-dd-HH-MM-ss');
let path = outputUriPrefix;
if (path.endsWith('/')) {
path += timestamp;
} else {
path += '/' + timestamp;
}
const body = {
outputUriPrefix: path,
};
// If specified, mark specific collections for backup
const { collections } = req.query;
if (collections) {
body.collectionIds = collections.split(',');
}
const projectId = process.env.GOOGLE_CLOUD_PROJECT;
const url = 'https://firestore.googleapis.com/v1beta1/projects/' + projectId + '/databases/(default):exportDocuments';
try {
const response = await axios.post(url, body, { headers });
res
.status(200)
.send(response.data)
.end();
} catch (e) {
if (e.response) {
console.warn(e.response.data);
}
res
.status(500)
.send('Could not start backup:' + e.message)
.end();
}
});
°°°°
// Start the server
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log('App listening on port ${PORT}');
console.log('Press Ctrl+C to quit.');
});
I have another function that is listening to '/'. Is possible that this can cause the problem?
package.json:
{
"name": "solution-scheduled-backups",
"version": "1.0.0",
"description": "Scheduled Cloud Firestore backups via AppEngine cron",
"main": "app.js",
"engines": {
"node": "10.x.x"
},
"scripts": {
"deploy": "gcloud app deploy --quiet app.yaml cron.yaml",
"start": "node app.js"
},
"author": "Google, Inc.",
"license": "Apache-2.0",
"dependencies": {
"#google-cloud/storage": "^3.2.1",
"#google/maps": "^0.5.5",
"axios": "^0.19.0",
"dateformat": "^3.0.3",
"express": "^4.17.1",
"firebase-admin": "^8.4.0",
"googleapis": "^42.0.0",
"request": "^2.88.0"
},
"devDependencies": {
"prettier": "^1.18.2"
}
}
I also look inside Cron log and there is nothing related to the error. It only returns 500 error
The main problem is the bucket location
When you create a backup bucket you must use Multi-Region or you will receive a deny from the server.
I think this is a bug of Google Cloud
Solution
Delete the bucket and create a new one with a Multi-Regional location
The error with Single Location:
'Bucket backup-bucket is in location EUR4. This project can only operate on buckets
spanning location europe-north1 or europe-west1 or eu or europe-west2 or
europe-west3 or europe-west4 or europe-west5 or europe-west6.',
So i've been replicating your issue and i've found the solution.
I kept getting the same error as you and I finally managed to figure it out. If you look into GAE logs you can see the error saying that 'Project \' "YOUR PROJECT" \' is not a Cloud Firestore enabled project.'.
This worked for me:
Make a new project.
Go to API Library in GCP and enable Firestore Installation API.
Go to Firebase and link your GCP project to a firebase project.
Go to database and create Firestore Database.
Follow the repo with the permissions and the deployment to app engine.
Test the cron and it will be successful.
If you ever had DataStore enabled in your actual project you will not be able to do a Firestore instance at step 3.
This will create you the needed buckets with the .appspot.com format, that you have give permissions to.
Go to GAE and create your cron.yaml, app.js and everything else you need. I used this repo for tests.
In the readme.md of the repo you have the exact commands that you have to do in order to give permissions to your service account.
Remember to change the bucket as told in cron.yaml.
Follow the steps mentioned in the repo as they are pretty well done.
Let me know if it worked for you!
I am working on a React app created with create-react-app. I was having trouble creating a web worker in it so I posted a question here on SO: Creating a web worker inside React
I've found a solution, as written in the post above, to load a worker without ejecting the app and messing with the Webpack config. This is the code, from the post above:
// worker.js
const workercode = () => {
self.onmessage = function(e) {
console.log('Message received from main script');
const workerResult = 'Received from main: ' + (e.data);
console.log('Posting message back to main script');
self.postMessage(workerResult);
}
};
let code = workercode.toString();
code = code.substring(code.indexOf("{")+1, code.lastIndexOf("}"));
const blob = new Blob([code], {type: "application/javascript"});
const worker_script = URL.createObjectURL(blob);
module.exports = worker_script;
and in the file that uses the worker:
import worker_script from './worker';
const myWorker = new Worker(worker_script);
myWorker.onmessage = (m) => {
console.log("msg from worker: ", m.data);
};
myWorker.postMessage('im from main');
It works, however, I cannot seem to get importScripts to work. Even if I do this (outside onmessage or inside onmessage):
if (typeof importScripts === 'function') {
importScripts('myscript.js');
}
In that case, the if statement turns out to be true, but then fails on the actual import with the same error message 'importScripts' is not defined as if the if statement is a false positive, which doesn't sound right. I'd say this is a context issue and that the worker probably isn't loading properly (although it seems to work), but it's just a guess.
Any ideas what's happening here?
importScripts in a worker created from Blob works fine, at least in 2021 (react 17.0.2, react-scripts 4.0.3, Chrome 92). The imported script URL must be absolute because worker was created from Blob.
The original issue might have been a bug in webpack or the transpilation might have changed the code in a weird way.
const workercode = () => {
importScripts("https://example.com/extra.js");
console.log(self.extraValue); // 10
self.onmessage = function(e) {
console.log('Message received from main script');
...
}
};
// extra.js
self.extraValue = 10;
Looks like this is still broken in 2022 - Seems there is a regression coming down the dev pipeline (at least in Android WebView and possibly some dev/canary chrome verions.)
https://bugs.chromium.org/p/chromium/issues/detail?id=1078821