I have uploaded the model.json file of my tensorflow graph to a private repository on an AWS S3 bucket, and am now trying to load the graph with the loadGraphModel (alongside with the binary files of the weight manifest values, group1-shard1of1). Here's my code, which I run with node (I've kept the bucket path and signature keys private)
TFJSConverter = require('#tensorflow/tfjs-converter')
const MODEL_URL = "https://[BucketName].s3.amazonaws.com/[PathToModel]/model.json?[credentials]&[securitykey]";
global.fetch = require('node-fetch')
TFJSConverter.loadGraphModel(MODEL_URL)
However the loadGraphModel function looks for a model url ending with '.json'. If not, it looks for the full model url and checks for a weight manifest file called weights_manifest.json, with no signature. An error request then follows:
UnhandledPromiseRejectionWarning: Error: Request to https://[BucketName].s3.amazonaws.com/[PathToModel]/model.json?[credentials]&[securitykey],https://[BucketName].s3.amazonaws.com/[PathToModel]/weights_manifest.json failed with status code 403. Please verify this URL points to the model JSON of the model to load.
I've checked that the signed url actually works, is there a solution for signed urls?
Installed versions:
#tensorflow/tfjs-converter#1.1.2
node v10.15.3
Many thanks!
The correct library to use to load the model is tfjs and not tfjs-converter
let tf = require("#tensorflow/tfjs");
tf.loadGraphModel(MODEL_URL)
403 error is an authorization error response. Try to set the credentials in the request using requestInit of the object passed as parameter of loadGraphModel
This worked for me:
const fetch = require('node-fetch')
global.fetch = fetch
but you can also try:
const fetch = require('node-fetch')
tf.loadGraphModel(MODEL_URL, { fetchFunc: fetch } )
as described in the documentation:
https://js.tensorflow.org/api/latest/#loadGraphModel
Related
So, I looked for a few authentication options for Next.js that wouldn't require any work on the server side of things. My goal was to block users from entering the website without a password.
I've set up a few tests with NextAuth (after a few other tries) and apparently I can block pages with sessions and cookies, but after a few hours of research I still can't find how I would go about blocking assets (e.g. /image.png from the /public folder) from non-authenticated requests.
Is that even possible without a custom server? Am I missing some core understanding here?
Thanks in advance.
I did stumble upon this problem too. It took my dumbass a while but i figured it out in the end.
As you said - for auth you can just use whatever. Such as NextAuth.
And for file serving: I setup new api endpoint and used NodeJS magic of getting the file and serving it in pipe. It's pretty similar to what you would do in Express. Don't forget to setup proper head info in your response.
Here is little snippet to demonstrate (typescript version):
import { NextApiRequest, NextApiResponse } from 'next'
import {stat} from "fs/promises"
import {createReadStream, existsSync} from "fs"
import path from "path"
import mime from "mime"
//basic nextjs api
export default async function getFile (req: NextApiRequest, res: NextApiResponse) {
// Dont forget to auth first!1!!!
// for this i created folder in root folder (at same level as normal nextjs "public" folder) and the "somefile.png" is in it
const someFilePath = path.resolve('./private/somefile.png');
// if file is not located in specified folder then stop and end with 404
if (! existsSync(someFilePath)) return res.status(404);
// Create read stream from path and now its ready to serve to client
const file = createReadStream(path.resolve('./private/somefile.png'))
// set cache so its proper cached. not necessary
// 'private' part means that it should be cached by an invidual(= is intended for single user) and not by single cache. More about in https://stackoverflow.com/questions/12908766/what-is-cache-control-private#answer-49637255
res.setHeader('Cache-Control', `private, max-age=5000`);
// set size header so browser knows how large the file really is
// im using native fs/promise#stat here since theres nothing special about it. no need to be using external pckages
const stats = await stat(someFilePath);
res.setHeader('Content-Length', stats.size);
// set mime type. in case a browser cant really determine what file its gettin
// you can get mime type by lot if varieties of methods but this working so yay
const mimetype = mime.getType(someFilePath);
res.setHeader('Content-type', mimetype);
// Pipe it to the client - with "res" that has been given
file.pipe(res);
}
Cheers
I am trying to use the searchKitManager inside react-admin I provided the parameters etc according to the docs but when I run the code it throws errors. Here is how the code works
React Admin is running on http://localhost:3000
Golang backend is running on http://localhost:3006
ElasticSearch is running on http://localhost:9200
When data is inserted in mysql database using golang code it is also inserted in elasticsearch later on in one of my display component I call the above searchkitManager as follows
let apiUrl= 'http://localhost:9200/donate' // what link should I pass, url to elasticsearch or url to my backend
const searchkit = new SearchkitManager('/', {
searchUrlPath: `${apiUrl}/_search`,
});
This code will throw 404 Not Found or 400 Bad Request error but the API works in postman
if I change the above link to
let apiUrl= 'http://localhost:9200/donate' // what link should I pass, url to elasticsearch or url to my backend
const searchkit = new SearchkitManager('/', {
searchUrlPath: `${apiUrl}/_doc/`,
});
I am not getting anything at all sometimes it no error in console and sometimes 400 Bad Request or 405 Post Not Allowed
One last thing the link I am providing as for searchUrlPath should be like that or not? or should I pass the apiUrl in place of /? I tried that as well but just to make sure.
Any kind of help will be really appreciated.
Try doing this:
const elasticSearchUrl = 'http://localhost:9200/<your_index>';
const searchkit = new SearchkitManager(elasticSearchUrl );
The front end enables people to upload their photos, so i was sending the base64 to the server and working with it initially, but there are problems with firewall which blocks the request which contains base64. As an alternative solution I was trying to upload the image to azure blob get the file name and then send that to the server for processing where I generate a sas token for the blob validation and processing.
This works perfectly fine when I work locally and the front end connection works with #azure/storage-blob
and uploadBrowserData() when I send the arrayBuffer as the param
export const uploadSelfieToBlob = async arrayBuffer => {
try {
const blobURL = `https://${accountName}.blob.core.windows.net${sasString}`;
const blobServiceClient = new BlobServiceClient(blobURL, anonymousCredential);
const containerClient = blobServiceClient.getContainerClient(containerName);
let randomString = Math.random().toString(36).substring(7);
const blobName = `${randomString}_${new Date().getTime()}.jpg`;
const blockBlobClient = containerClient.getBlockBlobClient(blobName);
const uploadBlobResponse = await blockBlobClient.uploadBrowserData(arrayBuffer);
return { blobName, blobId: uploadBlobResponse.requestId };
} catch (error) {
console.log('error when uploading to blob', error);
throw new Error('Error Uploading the selfie to blob');
}
};
When I deploy this is not working, the front is deployed in the EastUs2 location and the local development location is different.
I thought the sasString generated for anonymous access had the timezone option so I generated 2 different one's one for local and one for hosted server with the same location selected.
Failed to send request to https://xxxx.blob.core.windows.net/contanainer-name/26pcie_1582087489288.jpg?sv=2019-02-02&ss=b&srt=c&sp=rwdlac&se=2023-09-11T07:57:29Z&st=2020-02-18T00:57:29Z&spr=https&sig=9IWhXo5i%2B951%2F8%2BTDqIY5MRXbumQasOnY4%2Bju%2BqF3gw%3D
What am I missing any lead would be helpful thanks
First, as mentioned in the comments there was an issue with the CORS Settings because of which you're getting the initial error.
AuthorizationResourceTypeMismatchThis
request is not authorized to perform this operation using this
resource type. RequestId:7ec96c83-101e-0001-4ef1-e63864000000
Time:2020-02-19T06:57:31.2867563Z
I looked up this error code here and then closely looked at your SAS URL.
One thing I noticed in your SAS URL is that you have set the signed resource type (srt) as c (container) and trying to upload the blob. If you look at the description of the kind of operations you can do using srt=c here, you will notice that blob related operations are not supported.
In order to perform blob related operations (like blob upload), you would need to set signed resource type value to o (for object).
Please regenerate your SAS Token and include signed resource type as object (you can also include container and/or service in there as well) and then your request should work. So essentially your srt in your SAS URL should be something like srt=o or srt=co or srt=sco.
I couldn't notice anything wrong with the code you mentioned about, but I have been using a different method to upload files to Azure Blog Storage using React, the method is exactly the same as in this blog article which works perfectly for me.
https://medium.com/#stuarttottle/upload-to-azure-blob-storage-with-react-34f37805fdfc
In the dev environment for my React app, I have a set of public/private keys that I need to access an API. I'd like to ideally put these keys into their own file for gitignore purposes, but i'm not having luck with my code as shown below.
my helpers.jsx file is where the API data is called via lightweight AJAX add-on, and I have the actual keys in the require declarations area:
var API_KEY = require('./keys.jsx');
var PRIV_KEY = require('./keys.jsx');
Summarily, my keys.jsx file (stored in the same subfolder as the helpers.jsx) consists of the following:
module.exports = {
API_KEY:'myactualpublickey',
PRIV_KEY:'myactualprivatekey'
};
However, my app does not like this set up, as I get an "Failed to load resource: the server responded with a status of 401 (Unauthorized)” error message and the API call isn't successful because the necessary keys are not included.
When I replace the require('./keys.jsx'); in the helpers.jsx file with the actual keys, the API call works fine.
Any help or guidance would be most appreciated. Thanks.
You're exporting an object with properties called API_KEY and PRIV_KEY, so try this:
var API_KEY = require('./keys.jsx').API_KEY;
var PRIV_KEY = require('./keys.jsx').PRIV_KEY;
Following is my App Engine Endpoint. I annotate it as ApiMethod.HttpMethod.GET because I want to be able to make a get call through the browser. The class itself has a few dozen methods understandably. Some of them using POST. But getItems is annotated with GET. When I try to call the url through a browser, I get a 405 error
Error: HTTP method GET is not supported by this URL
The code:
#Api(name = "myserver",
namespace = #ApiNamespace(ownerDomain = "thecompany.com", ownerName = "thecompany", packagePath = ""),
version = "1", description = "thecompany myserver", defaultVersion = AnnotationBoolean.TRUE
)
public class myserver {
#ApiMethod(name = "getItems", httpMethod = ApiMethod.HttpMethod.GET)
public CollectionResponse<Item> getItems(#Named("paramId") Long paramId) {
…
return CollectionResponse.<Item>builder().setItems(ItemList).build();
}
}
This is not for localhost, it’s for the real server. Perhaps I am forming the url incorrectly. I have tried a few urls such as
https://thecompanymyserver.appspot.com/_ah/spi/com.thecompany.myserver.endpoint.myserver.getItems/v1/paramId=542246400
https://thecompanymyserver.appspot.com/_ah/spi/myserver/NewsForVideo/v1/542246400
The proper path for this is /_ah/api/myserver/1/getItems. /_ah/spi refers to the backend path, which only takes POST requests of a different format.
Side note: API versions are typical "vX" instead of just "X".
You can use the api explorer to find out whether you're using the correct url. Go to
https://yourprojectid.appspot.com/_ah/api/explorer
this works on the devserver as well:
http://localhost:8080/_ah/api/explorer
Also if you're not planning to use the google javascript api client you should add path="..." to your #ApiMethods, so you are sure about what the path actually is.