I am trying to load Tensorflow.js model via HTTP protocol. Tensorflow.js requires me to store 'model.json' and 'weights.bin' files in the same folder. But I can only call 'model.json' as a parameter. It refers to the binary file by itself. That is how it works as far as I know.
For now, in the local environment, I am loading the model from the localhost(Http://127.0.0.1:8080) and it works fine.
However, the actual application accepts HTTPS protocol only. So I have tried to store them with models and weights in the same buckets in S3 and called via Lambda but it seems like only 'model.json' is retrieved. I am thinking of using EC2 instances where the Python Flask server is running but it seems like the same that only model.json is retrieved, not binary files.
Is there any way that I can retrieve 'model.json' with referring to the weight file? Is there anyway to host file server remotely with HTTPS protocol?
TFJS downloads model JSON, parses it and uses whatever paths are specified in the JSON - you can edit that file and set any URL you want for weights.
Alternatively, you can also use lower-level methods to load weights manually (in case you want to have a custom loader, etc.), but leave that for future until you're more comfortable with TFJS.
Related
I'm developing a single page app for image annotation. Each .jpg file is stored on S3/minIO services, coupled with a .xml file (Pascal VOC notation), which describes the coordinates and positions for each annotation associated to the image.
I'd like to fetch all the xml data, to be able filtering my image results within the webapp project (based upon ReactJS). But thousand of request to an S3 server directly from a web app seems a bit odd to me; nevertheless, I would prefer avoid using any "middleware" servers (like python/flask or nodejs), relying on the ReactJS app.
I've not been able to find any workaround to download all the xml files content with a single ajax call; do you have some idea to address this kind of issue?
The S3 API doesn't provide an API to fetch multiple files in a single operation. As you have suggested in your question, your application will need to handle this logic by first getting a list of the objects then iterating through that list.
Alternatively, if you can consider storing the xml files as a single archive.
Please presume that I do not know anything about any of the things I will be mentioning because I really do not.
Most OpenData sites have the possibility of exporting the presented file either in for example .csv or .json formats (Example). They also always have an API tab (Example API).
I presume using the API would mean that if the data is updated you would receive the change whereas exporting it as .csv would mean the content will not be changed anymore.
My questions is: how does one use this API code to display the same table one would get when exporting a .csv file.
Would you use a database to extract this information? What kind of database and how do you link the API to the database?
I presume using the API would mean that if the data is updated you
would receive the change whereas exporting it as .csv would mean the
content will not be changed anymore.
You are correct in the sense that, if you download the csv to your computer, that csv file won't be updated any more.
An API is something you would call - in this case, you can call the API, saying "Hey, do you have the latest data on xxx?", and you will be given back the latest information about what you have asked. This does not mean though, that this site will notify you when there's a new update - you will have to keep calling the API (every hour, every day etc) to see if there are any changes.
My questions is: how does one use this API code to display the same
table one would get when exporting a .csv file.
You would:
Call the API from a server code, or a cloud service
Let the server code or cloud service decipher (or "Parse") the response
Use the deciphered response to create a table made out of HTML, or to place it into a database
Would you use a database to extract this information? What kind of
database and how do you link the API to the database?
You wouldn't necessarily need a database to extract information, although a database would be nice to place the final data inside.
You would first need some sort of way to "call the REST API". There are many ways to do this - using Shell Script, using Python, using Excel VBA etc.
I understand this is hard to visualize, so here is an example of step 1, where you can retrieve information.
Try placing in the below URL (taken from the site you showed us) in your address bar of your Chrome browser, and hit enter
http://opendata.brussels.be/api/records/1.0/search/?dataset=associations-clubs-sportifs
See how it gives back a lot of text with many brackets and commas? You've basically asked the site to give you some data, and this is the response they gave back (different browsers work differently - IE asks you to download the response as a .json file). You've basically called an API.
To see this data more cleanly, open your developer tools of your Chrome browser, and enter the following JavaScript code
var url = 'http://opendata.brussels.be/api/records/1.0/search/?dataset=associations-clubs-sportifs';
var xhr = new XMLHttpRequest();
xhr.open('GET', url);
xhr.setRequestHeader('X-Requested-With', 'XMLHttpRequest');
xhr.onload = function() {
if (xhr.status === 200) {
// success
console.log(JSON.parse(xhr.responseText));
} else {
// error
console.log(JSON.parse(xhr.responseText));
}
};
xhr.send();
When you hit enter, a response will come back, stating "Object". If you click through the arrows, you can see this is a cleaner version of the data we just saw - more human readable.
In this case, I used JavaScript to retrieve the data, but you can use whatever code you want. You could proceed to use JavaScript to decipher the data, manipulate it, and push it into a database.
kintone is an online cloud database where you can customize it to run JavaScript codes, and have it store the data in their database, so you'll have the data stored online like in the below image. This is just one example of a database you can use.
There are other cloud services which allow you to connect API end points of different services with each other, like IFTTT and Zapier, but I'm not sure if they connect with open data.
The page you linked to shows that the API returns values as a JSON object. To access the data you can just send an appropriate http request and the response will be the requested data as a JSON. You can send requests like that over your browser if you want to.
Most languages allow JSON objects to be manipulated pro grammatically if you need to do work on the data.
Restful APIs publish model is "request and publish". Wen you request data via an API endpoint, you would receive response strings in JSON objects, CSV tables or XML.
The publisher, in this case Opendata.brussel.be would update their database on regular basis and publish the results via an API endpoint.
If you want to download the table as a relational data table in a CSV file, you'd need to parse the JSON objects into relational tables. This can be tricky since each JSON response string can vary in their paths.
There're several ways to do it. You can either write scripts to flatten the JSON objects or use a tool to parse and flatten the objects for you.
I use a tool called Acho to turn API endpoints into CSV files. It would parse almost all API endpoints through the parameters and even configure for multiple requests, such as iterative and recursive requests.
Acho API parser
I search how to write a file with angularjs, after get it in json. I have already the json file in my folder, but with onclick, users can manually update it with the new content and overwrite the same file. How to write it ? (i don't ask the get distant file part).
I need locastorage functions ? it's for android application, so which storage i must use ?
You can't write files with javascript. You would need to pass the data to PHP or nodejs (aka a server side language) that has access to your filesystem. Javascript is client facing, so it doesn't know anything about the filesystem on the server it is running.
Is it possible to use a local database file with html5 without using a server. I would like to create a small application that depends on information from a small database. I do not want to host a server just to pull information. Is it possible to create a database file and pull information from the local files ?
Depends on the following:
The type of application you want to build:
Normal website with some data being pulled from a local storage;
Special purpose hosted website / application with data generated by the user;
Special purpose local application with a dedicated platform (a particular browser) and with access to the browser's non-web API -- in order to access the browser's own persistent storage methods (file storage, SQLite etc.);
Special purpose local application with a dedicated environment -- in order to deploy the application with a local web server and database;
Available options:
Indexed DB
Web Storage
XML files used for storing data and XSLT stylesheets for translating the data into HTML;
Indexed DB and Web Storage ar available in some browsers but you need to make sure the targeted browsers have it. Their features aren't quite as complete and flexible as SQL RDBMSs but they may fit the bill if your application doesn't need all that flexibility.
XML files can contain the data you want to be shown to the user and they can be updated manually (not by the user) or dynamically (by a server script).
For dynamic updating the content of the XML is kept in JavaScript and manipulated / altered (using the XML DOM) and when the session is over the XML content is sent to the server to entirely replace the previous XML file. This works OK if the individual users have a file each and they never write to each other's files.
Reading local files:
Normal file access is prohibited (for security reasons) to all local (JavaScript) code, which means that "having" a file locally implies either downloading it from a known source (a server) or asking the user to offer access to a local file.
Asking the user to offer access to a local file which implies offering the user a "file input" -- like for uploads but without actually uploading the file.
After a file has been selected using FileAPI to read that file should be fairly simple.
This workflow would involve the user "giving" you the database on every page refresh -- but since it's a one page thing it would mean giving you the data on every session as long as your script does not refresh the page.
You can use localstorage but you can run a server from your own computer. You can use Wamp or Xampp. Which use Apache and mysql.
What i'm looking for is a little more robust than a cookie. I am making a web application for a friend that will be 1 page, and have a list of names on the page. The person wants to be able to add names to the list, however they do not want to use a web server. Just want the files locally on a computer so a folder called test-app , with index.html, and possibly a database file that can be stored in the web browser or a way to save information to the web browser for repeated use.
I'm trying to develop AngularJS applicatino using the Angular tutorial web-server script.
Is it possible or smart to use it for development only scenario ?
I want to be able to develop and test my Angular application without relying on the real server and real database, that's the reason I'm asking this.
I don't know much about the tutorial web-server script.
When it comes to your situation, though, your best bet is to abstract away your data managing processes. In other words, you can make a set of services that take care of loading and saving your data. You could have methods like book.save() or book.fetch().
Then in save() and fetch() you can return or insert an object literal or call for a JSON file.
Assuming that your product will be running on JSON data, you should be able to write another set of model services that call JSON data from the server rather any that you've hard written in the code or in a *.json file.