I put this code in Heroku but for some reason is not working.
This is my code:
Client.on('ready', async () => {
await connect(config.MongoPath, {
useNewUrlParser: true,
useUnifiedTopology: true,
});
console.log("Ready!")
})
This is my schema:
const { Schema, model } = require('mongoose');
const PendingList = Schema({
id: String,
PendingList: {
default: [],
type: Array
}
});
module.exports = model('PendingList',PendingList);
I am receiving this error in Heroku.
It works perfectly on my local machine but not Heroku.
That's because your heroku app does not have the permissions to access your database cluster.
You need to go to your Mongo Atlas cluster Log in here and then whitelist heroku's IP inorder for the servers to access your DB
navigate to security > network access > and add this IP
Related
I am having the same issue as issue CORs Error: Google Oauth from React to Express (PassportJs validation). But I am unable to get the solution offered by #Yazmin to work.
I am attempting to create a React, Express/Nodejs, MongoDB stack with Google authentication and authorization. I am currently developing the stack on Windows 10, using Vs Code (React on ‘localhost:3000, Nodejs on localhost:5000 and MongoDB on localhost:27017.
The app’s purpose is to display Urban Sketches(images) on a map using google maps, google photos api and google Gmail api. I may in the future also require similar access to Facebook Groups to access Urban Sketches. But for now I have only included the profile and Email scopes for authorization.
I want to keep all requests for third party resources in the backend, as architecturally I understand this is best practice.
The google authorization process from origin http://localhost:5000 works just fine and returns the expected results. However, when I attempt to do the same from the client - origin Http://localhost:3000 the following error is returned in the developers tools console following the first attempt to access the google auth2 api. Although the scheme and domain are the same the port is different, so the message from the third part (Https://account.google.com) has been rejected by the browser.
Access to fetch at 'https://accounts.google.com/o/oauth2/v2/auth?response_type=code&redirect_uri=http%3A%2F%2Flocalhost%3A5000%2Fauth%2Fgoogle%2Fcallback&scope=profile%20email%20https%3A%2F%2Fmail.google.com%2F&client_id=' (redirected from 'http://localhost:3000/auth/google') from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
No matter what I try the error message is the same.
I think that google is sending the reply to the client (localhost:3000) rather than to the server.
Among other solutions, I attempted to implement Yilmaz’s solution by Quote: “Create setupProxy.js file in client/src. No need to import this anywhere. create-react-app will look for this directory” I had already created my client by running create-react-app previously. So I added setupProxy.js inside my src folder.
Question: I assume I am correct that the new setupProxy.cjs file containing my settings will be included by webpack after I restart the client.
It seems to me that the flow I am getting is not BROWSER ==> EXPRESS ==> GOOGLE-SERVER but BROWSER ==> EXPRESS ==> GOOGLE-SERVER ==>BROWSER where it stops with the cors error as shown above.
To test this theory, I put some console log messages in the client\node_modules\http-proxy-middleware\lib\index.js functions "shouldProxy" and "middleware", but could not detect any activity from the auth/google end point from the google authorization server response (https://accounts.google.com/o/oauth2/v2/auth).
So I think my theory is wrong and I don't know how I will get this working.
Console log messages displayed on VsCode terminal following request to /auth/google endpoint from the React client are as follows...
http-proxy-middleware - 92 HttpProxyMiddleware - shouldProxy
context [Function: context]
req.url /auth/google
req.originalUrl /auth/google
Trace
at shouldProxy (C:\Users\User\github\GiveMeHopev2\client\node_modules\http-proxy-middleware\lib\index.js:96:13)
at middleware (C:\Users\User\github\GiveMeHopev2\client\node_modules\http-proxy-middleware\lib\index.js:49:9)
at handle (C:\Users\User\github\GiveMeHopev2\client\node_modules\webpack-dev-server\lib\Server.js:322:18)
at Layer.handle [as handle_request] (C:\Users\User\github\GiveMeHopev2\client\node_modules\express\lib\router\layer.js:95:5)
at trim_prefix (C:\Users\User\github\GiveMeHopev2\client\node_modules\express\lib\router\index.js:317:13)
at C:\Users\User\github\GiveMeHopev2\client\node_modules\express\lib\router\index.js:284:7
at Function.process_params (C:\Users\User\github\GiveMeHopev2\client\node_modules\express\lib\router\index.js:335:12)
at next (C:\Users\User\github\GiveMeHopev2\client\node_modules\express\lib\router\index.js:275:10)
at goNext (C:\Users\User\github\GiveMeHopev2\client\node_modules\webpack-dev-middleware\lib\middleware.js:28:16)
at processRequest (C:\Users\User\github\GiveMeHopev2\client\node_modules\webpack-dev-middleware\lib\middleware.js:92:26)
http-proxy-middleware - 15 HttpProxyMiddleware - prepareProxyRequest
req localhost
The Google callback uri is http://localhost:5000/auth/google/callback
This is a listing of my nodejs server code.
dotenv.config();
// express
const app = express();
// cors
app.use(cors())
// passport config
require ('./config/passport')(passport)
// logging
if( process.env.NODE_ENV! !== 'production') {
app.use(morgan('dev'))
}
const conn = process.env.MONGODB_LOCAL_URL!
/**
* dbConnection and http port initialisation
*/
const dbConnnect = async (conn: string, port: number) => {
try {
let connected = false;
await mongoose.connect(conn, { useNewUrlParser: true, useUnifiedTopology: true })
app.listen(port, () => console.log(`listening on port ${port}`))
return connected;
} catch (error) {
console.log(error)
exit(1)
}
}
const port = process.env.SERVERPORT as unknown as number
dbConnnect(conn, port)
//index 02
// Pre Middleware
app.use(express.json());
app.use(express.urlencoded({ extended: true }))
const mongoStoreOptions = {
mongoUrl: conn,
collectionName: 'sessions'
}
app.use(
session({
secret: process.env.SESSIONKEY as string,
resave: false,
saveUninitialized: false,
store: MongoStore.create(mongoStoreOptions),
})
)
app.use(passport.initialize())
app.use(passport.session())
// Authentication and Authorisation
const emailScope: string = process.env.GOOGLE_EMAIL_SCOPE as string
//GOOGLE_EMAIL_SCOPE=https://www.googleapis.com/auth/gmail/gmail.compose
const scopes = [
'profile',
emailScope
].join(" ")
app.get('/auth/google', passport.authenticate('google', {
scope: scopes
}));
app.get('/auth/google/callback', passport.authenticate('google', { failureRedirect: '/'}),
(req, res) => {
res.send('Google Login Successful ')
}
)
app.get('/', (req, res) => {
res.send('Hello World');
})
The http-proxy-middleware setupProxy.cjs file. Note the cjs extension. I assume this was because I am using Typescript. It is in the client src folder
const createProxyMiddleware = require('http-proxy-middleware');
module.exports = function (app) {
app.use(createProxyMiddleware('/auth', {target: 'http://localhost:5000'}))
}
And finally the fetch command from the client
async function http(request: RequestInfo): Promise<any> {
try {
const response = await fetch('/auth/google')
const body = await response.json();
return body
} catch (err) { console.log(`Err SignInGoogle`) }
};
And the passport config...
import { PassportStatic} from 'passport';
import {format, addDays} from 'date-fns'
import { IUserDB, IUserWithRefreshToken, ProfileWithJson} from '../interfaces/clientServer'
const GoogleStrategy = require('passport-google-oauth20').Strategy;
const User = require('../models/User')
module.exports = function (passport:PassportStatic) {
const clientID: string = process.env.GOOGLE_CLIENTID as string
const clientSecret: string = process.env.GOOGLE_SECRET as string
const callbackURL: string = process.env.GOOGLE_AUTH_CALLBACK as string
const strategy = new GoogleStrategy(
{
clientID: clientID,
clientSecret: clientSecret,
callbackURL: callbackURL,
proxy: true
},
async (_accesstoken: string, _refreshtoken: string,
profile: ProfileWithJson,
etc
you can't make a fetch call to the /auth/google route!
Here's my solution in javascript...
// step 1:
// handler function should use window.open instead of fetch
const loginHandler = () => window.open("http://[server:port]/auth/google", "_self")
//step 2:
// on the server's redirect route add this successRedirect object with correct url.
// Remember! it's your clients root url!!!
router.get(
'/google/redirect',
passport.authenticate('google',{
successRedirect: "[your CLIENT root url/ example: http://localhost:3000]"
})
)
// step 3:
// create a new server route that will send back the user info when called after the authentication
// is completed. you can use a custom authenticate middleware to make sure that user has indeed
// been authenticated
router.get('/getUser',authenticated, (req, res)=> res.send(req.user))
// here is an example of a custom authenticate express middleware
const authenticated = (req,res,next)=>{
const customError = new Error('you are not logged in');
customError.statusCode = 401;
(!req.user) ? next(customError) : next()
}
// step 4:
// on your client's app.js component make the axios or fetch call to get the user from the
// route that you have just created. This bit could be done many different ways... your call.
const [user, setUser] = useState()
useEffect(() => {
axios.get('http://[server:port]/getUser',{withCredentials : true})
.then(response => response.data && setUser(response.data) )
},[])
Explanation....
step 1 will load your servers auth url on your browser and make the auth request.
step 2 then reload the client url on the browser when the authentication is
complete.
step 3 makes an api endpoint available to collect user info to update the react state
step 4 makes a call to the endpoint, fetches data and updates the users state.
Using knex in a Nuxt app to query a SQL Server database hosted on Azure. When querying one particular table with ~150k rows, the app crashes but does print the length of the results returned. When querying a smaller table with ~2k rows, there is no problem.
Is there a limitation on how much data I can return from a single query? I need to be able to return about 1 million rows of data across several tables so that I can aggregate and display some calculations done with the raw table data.
I think it's understandable that the amount of data takes up too much memory, but I would like to know if there is any workaround to returning tons and tons of rows w/o issue.
api/routes/tickets.js
const { Router } = require('express');
const router = Router();
const knex_db = require('knex')({
client: 'mssql',
connection: {
host: 'mydb.database.windows.net',
user: 'user',
password: 'secret',
port: 1433,
options: {
database: 'mydatabase',
encrypt: true
}
}
});
router.get('/tickets/all', async function(req, res) {
const results = await knex_db('dbo.tickets');
console.log('results.length: ' + results.length);
res.json({data: results});
})
module.exports = router;
api/index.js
const express = require('express');
const app = express();
const tickets = require('./routes/tickets');
app.use(tickets);
module.exports = {
path: '/api',
handler: app
}
pages/setup/index.vue
<script>
export default {
async asyncData ({ $axios }) {
const data = (await $axios.$get('/api/tickets/all')).data;
// console.log(data);
return { tickets: data }
}
}
</script>
I was able to resolve this issue by changing my code from
res.json(...)
to
res.status(200).json(...)
For some reason res.json must have been causing a memory leak or something of the sorts.
I am developing a typical MERN application and I've completed the authentication cycle. My NodeJS/Express back-end uses 'express-session' and 'connect-mongodb-connection' to create and handle sessions. The React front-end uses 'axios' for communicating with the API. The authentication cycle works on all browsers except Chrome. For all other browsers, a session is successfully created in MongoDB, cookies are set in the browser and I am successfully logged into a session.
But when testing this with Chrome, everything works perfectly except for the part where cookies are set. I've tested this rigorously over the span of a day and I can trace the cookie to the point where it's sent from the back-end. But Chrome refuses to save the cookie.
Here is my code for maintaining sessions:
server/app.js
var store = new MongoDBStore({
uri: DB,
collection: 'sessions'
});
// Catch errors
store.on('error', function (error) {
console.log(error);
});
app.use(require('express-session')({
secret: process.env.SESSION_SECRET,
saveUninitialized: false, // don't create session until something stored
resave: false, //don't save session if unmodified
store: store,
cookie: {
maxAge: parseInt(process.env.SESSION_LIFETIME), // 1 week
httpOnly: true,
secure: !(process.env.NODE_ENV === "development"),
sameSite: false
},
}));
//Mongo Session Logic End
app.enable('trust proxy');
// 1) GLOBAL MIDDLEWARES
// Implement CORS
app.use(cors({
origin: [
process.env.CLIENT_ORIGINS.split(',')
],
credentials: true,
exposedHeaders: ['set-cookie']
}));
The CLIENT_ORIGINS are set to the https://localhost:3000 and http://localhost:3000 where my React client runs.
Some things I've tried:
Trying all combinations of secure:true & secure:false with all combinations of sameSite:false & sameSite:'strict'
Setting domain to NULL or empty string
Trying to change path randomly
Here's my code for setting the cookies on login at the back-end:
exports.signIn = async (req, res, next) => {
const { email, password } = req.body;
if (signedIn(req)) {
res.status(406).json('Already Signed In');
return;
}
const user = await User.findOne({ email: email });
if (!user) {
res.status(400).json('Please enter a correct email.');
return;
}
if (!(await user.matchPassword(password))) {
res.status(400).json('Please enter a correct password.');
return;
}
req.session.userId = user.id;
res.status(200).json({ msg: 'Signed In', user: user });
};
This is the generic request model I use for calling my API from React using Axios:
import axios from "axios";
import CONFIG from "../Services/Config";
axios.defaults.withCredentials = true;
const SERVER = CONFIG.SERVER + "/api";
let request = (method, extension, data = null, responseTypeFile = false) => {
//setting up headers
let config = {
headers: {
"Content-Type": "application/json",
},
};
// let token = localStorage["token"];
// if (token) {
// config.headers["Authorization"] = `Bearer ${token}`;
// }
//POST Requests
if (method === "post") {
// if (responseTypeFile) {
// config['responseType'] = 'blob'
// }
// console.log('request received file')
// console.log(data)
return axios.post(`${SERVER}/${extension}`, data, config);
}
//PUT Requests
else if (method === "put") {
return axios.put(`${SERVER}/${extension}`, data, config);
}
//GET Requests
else if (method === "get") {
if (data != null) {
return axios.get(`${SERVER}/${extension}/${data}`, config);
} else {
return axios.get(`${SERVER}/${extension}`, config);
}
}
//DELETE Requests
else if (method === "delete") {
if (data != null) {
return axios.delete(`${SERVER}/${extension}/${data}`, config);
} else {
return axios.delete(`${SERVER}/${extension}`, config);
}
}
};
export default request;
Some more things that I have tested:
I have double checked that credentials are set to true on both sides.
I have made sure that the authentication cycle is working on other browsers.
I have also made sure that the authentication cycle works on Chrome when I run React on http instead of https
I have also added my self signed certificate into the trusted root certificates on my local machine. Chrome no longer shows me a warning but still refuses to save cookies
I have made sure that the authentication cycle works if I run an instance of Chrome with web security disabled.
I've tried to make it work by using 127.0.0.1 instead of localhost in the address bar to no avail.
No errors are logged on either side's console.
Any and all help would be appreciated
Chrome is always doing crazy stuff with cookies and localStorage...
It seems since chrome 80 chrome will reject any cookies that hasn't specifically set SameSite=None and Secure while using cross site requests. That issue, https://github.com/google/google-api-javascript-client/issues/561, is still open and being discussed there. I also think that using https while not setting Secure will also have it be rejected.
I have faced this same issue once and I have solved it by specifically set mentioned below:
document.cookie = "access_token=" + "<YOUR TOKEN>" + ";path=/;domain=."+ "<YOUR DOMAIN NAME>" +".com;secure;sameSite=none";
Make sure:
Your Path variable is set to /.
Your Domain is set to .<YOUR DOMAIN NAME>.com (NOTE: Here . dots is necessary part).
Your secure variable should be true.
Your sameSite variable should be none.
So I figured out the solution to my issue. My client-side was running on an https connection (even during development), because the nature of my project required so.
After much research, I was sure that the settings to be used for express-session were these:
app.use(require('express-session')({
secret: process.env.SESSION_SECRET,
saveUninitialized: false, // don't create session until something stored
resave: false, //don't save session if unmodified
store: store,
cookie: {
maxAge: parseInt(process.env.SESSION_LIFETIME), // 1 week
httpOnly: true,
secure: true,
sameSite: "none"
},
}));
Keep in mind that my client-side is running on an https connection even in development. However, despite using these settings, my login cycle did not work on Chrome and my cookies weren't being set.
Express session refused to send back cookies to the client, because despite having my client run on an https connection, it contacted my server on an http connection (my server was still running on an http connection in development), hence making the connection insecure.
So I added the following code to my server:
const https = require('https');
const fs = require('fs');
var key = fs.readFileSync("./certificates/localhost.key");
var cert = fs.readFileSync("./certificates/localhost.crt");
var credentials = {
key,
cert
};
const app = express();
const port = process.env.PORT || 3080;
const server = process.env.NODE_ENV === 'development' ? https.createServer(credentials, app) : app;
server.listen(port, () => {
console.log(`App running on port ${port}...`);
});
I used a self-signed certificate to run my server on an https connection during development. This along with sameSite: "none" and secure: true resolve the issue on Chrome (and all other browsers).
I am developing a website in NextJs and using MongoDB as a database. This code is what connects to the database and keeps it in cache.
import { MongoClient } from "mongodb";
let cache = {};
export default async function connect() {
if (cache?.client?.isConnected()) {
return cache;
}
const opts = {
useNewUrlParser: true,
useUnifiedTopology: true,
};
return MongoClient.connect(process.env.DATABASE_URL, opts).then((client) => {
cache = {
db: client.db("bd"),
client,
};
return {
client,
db: client.db("bd"),
};
});
}
I imagine that this code should cause only 1 connection to be created but it is creating many more connections. As in the photo below, 18 connections were created and dropped to 6 after I stopped use the site. Why are several connections being created? How do you make it just one?
This is an example of code that I am using on one of the routes to list users.
...
const { db, client } = await connect();
const { userThatMakeRequest } = req;
const { group } = req.query;
const users = await db
.collection("user")
.find(
{
roles: "team-user",
"team.id": userThatMakeRequest?.team?.id,
"group.id": group,
hasAccess: true,
},
{ projection: { password: 0 } }
)
.toArray();
res.status(200).json({
response: users || [],
});
The driver creates 1 or 2 connections to each known server for monitoring purposes. Application connections (the ones used for satisfying queries and writes) are separate.
If you create a client object and perform one query against a 3 node replica set running MongoDB 4.4, you'll end up with 7 total connections.
There's a dev database already set up for another project. I'm trying to create a sails.js server to connect to this database and act as a RESTful API. I'm also using SQL Workbench with the profile below to connect to the database and verify my query statements. On that tool, I'm able to send queries like select top 10 * from advisor and get the data I expect in response.
My connection configuration in sails.js seems to be alright, since I'm able to start the server. I've gotten simple static actions to work, like hi: function (req, res) { return res.send("Hi there!"); }. However, I can't figure out what to do to get a response from the database served by sails. My goal (at this point) is to have http://localhost:1337/advisor return JSON for the results of select top 10 * from advisor.
I initially tried using the freshly-generated model. Then, I tried adding attributes to the model file. Then, I tried adding my own code to the controller. In each case, the browser never received a response. At the end, I tested /advisor/list to run my own code and it doesn't look like the query() callback was ever executed. In case it's the first question, I have run npm install sails-sqlserver and I've double-checked that my host, db, username, & password are identical to what was used in Workbench.
connections.js
sqlserver: {
adapter: 'sails-sqlserver',
user: 'myusername',
password: 'mypassword',
host: 'mysubdomain.mydomain.net:1433',
database: 'frontofficedev'
}
models.js
module.exports.models = {
connection: 'sqlserver',
migrate: 'safe'
};
api\models\Advisor.js
module.exports = {
attributes: {
advcode: 'string',
advname: 'string',
'adv-default': 'boolean',
"user-id": 'string',
"pc-code": 'string',
"adv-tag": 'string',
"is-group": 'boolean',
"trade-grouping": 'string',
AdvisorId: 'int',
orgcode: 'string',
BranchId: 'int',
OrdPrnBranchId: 'int',
zdec1: 'float',
zdec2: 'float',
zchar1: 'string',
zchar2: 'string',
zchar3: 'string',
zchar4: 'string',
AdvStatus: 'string'
}
};
api\controllers
module.exports = {
hi: function (req, res) {
return res.send("Hi there!");
},
list: function (req, res) {
var myQuery = "select TOP 10 * from advisor";
sails.log.debug("Query :", myQuery);
console.log(Advisor);
Advisor.query(myQuery, function (err, advisors){
console.log(advisors);
console.log(err);
if(err || !advisors.rows.length){
return res.json({"status": 0, "error": err});
}
else{
return res.json(advisors);
}
});
}
};
Any ideas on what I'm doing wrong? Is JDBC causing problems? Thanks in advance.
Im assuming you've already run: npm install sails-sqlserver --save
You have to specify your connection and the table you will be using in the model, the variables in the model should match with your DB variables, like this:
api\models\Advisor.js
module.exports = {
schema: true,
connection: 'sqlserver',
tableName: 'yourTableName',
attributes: {
advcode:{
type: 'string',
primaryKey: true //if this is a primary key
},
advname:{
type: 'string'
},
'adv-default':{
type: 'boolean'
}
};
In your controller you can use the Sails ORM waterline like this:
api\controllers
module.exports = {
list: function (req, res) {
Advisor.query('SELECT * FROM advisor', function(err, results) {
if (err) {
res.send(400);
} else {
res.send(results);
}
});
}
};
Where Advisor is the model.
For more specific information about models and ORM waterline i recommend you read the sails docs: http://sailsjs.org/documentation/reference/waterline-orm/models
My colleague spotted the problem. The port that database lives on needs to be a separate attribute in sails' connection.js (instead of including it in the host string). No need for extra libraries, like node-jdbc.
config/connections.js
sqlserver: {
adapter: 'sails-sqlserver',
user: 'myusername',
password: 'mypassword',
host: 'mysubdomain.mydomain.net',
port: 1433,
database: 'frontofficedev'
}
After making that change, I was able to delete all my custom code from the controller and almost everything from the model (I still need to specify a primary key, since sails looks for id by default and the database was using AdvisorId.
api/models/Advisor.js
module.exports = {
attributes: {
AdvisorId: {primaryKey: true}
}
};