I am writing a simple filter function that sends a request and gets all entries that partially match the query. Desired behavior is if do a GET request localhost:3000/employees?email=mai I would like to receive all entries from a database starting with mai. Is there any way to construct an http requests for the desired behavior from a front-end part of the application? Perhaps some headers that tell the server that the search should not be strict? I have tryed googling it but failed. I am using AngularJS fro front-end and json-server as a back-end mock server
Up to my understanding if you filter data in front End you will get only that data filtered which you are showing. In the other hand If you do this using Query(Back End) all the data get filtered in DB.
Related
Right now I'm working on migrating a site from using an oracle database to use salesforce. For querying the data we are using the latest version of the salesforce rest API. Right now I'm facing a problem paginating results with an offset greater than 2000. I have seen quite a few questions on this topic but none of them seem to fit my problem.
So Here are the restrictions
I need to fetch the results on chunks of 20 records (that is the page size). I can't just get a bunch of results and then do some manual pagination because that would be out of scope for the project.
I can't do something like WHERE ID > lastIdIntheResults LIMIT 20 because I need to sort my results by Name, sorting by Id would break the order that the client expects.
Needs to be done using the REST API. I don't think there is a way to use queryMore function on the REST API.
So, do you have any suggestions?
Thanks
REST API's equivalent of SOAP API's queryMore is done: false and nextRecordsUrl: ... in the response.
https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_query.htm
Here's my data queried via https://workbench.developerforce.com -> Utilities -> Rest Explorer:
And then the next one would be
And to change the "chunk size" without LIMIT/OFFSET you can use a HTTP header: Sforce-Query-Options: batchSize=200
I have a React/Redux front-end with an Express back-end application, and I'm rather new, but I have a question regarding how to deal with the flow of data.
So, on my front end side, I have a search bar. When a user enters a search term, I sent a post request from React which is handled in my Express routes.js file. In this file, I am taking that search term, and I am looking for that term in my Mongo database. After that, all I want to do is send an object back if the term was found in the database.
I have used axios in this application to make an HTTP request to a certain route to pull off some data, but that was within an app.get(...) on the express side, and I used an axios.get(...) on the React side to retrieve the information.
But, this situation is slightly different since the data is flowing in two directions. Initially, from a front end to the backend, and then back-end to front-end. And in this case, I'm using app.post(...).
Now my question is, how would I retrieve the data to the front end? Could I simply just do an axios.get(...) on an app.post(...) or is there some other way to do this?
If you GET from the browser to your back-end's route which is implemented to respond to POST only, you will probably get a 405 error. Implement a POST Axios request and a POST Express reply.
You can use either GET or POST, but you need to be consistent on the server and the client side. If you do an http GET from the client, the server will only respond if you have a app.get(...) as a server route.
As far as the flow of data is concerned, both a get and a post can return data, it just needs to be specified on the express route.
After the business logic of looking if the key exists in mongo do something like res.send({'found': true}) or res.json({'found': false}). This will ensure that the data gets back to the client.
If I were to do this, I would:
1.) Use an Axios get request and pass in as a parameter the identifying attributes, such as a related _id or key phrase.
2.) Use mongoDB's query filter search parameters to index and aggregate the schema data in the DB. I would probably use .findOne or .find.
3.) Use the router callback to pass in the filtered data, then dispatch a function to save it to a state.
This way you can set up specific terms or keywords to search with, and utilize the searched data throughout the app.
I have a page with multiple widgets, each receiving data from a different query in the backend. Doing a request for each will consume the limit the browser puts on the number of parallel connections and will serialize some of them. On the other hand, doing one request that will return one response means it will be as slow as the slowest query (I have no apriori knowledge about which query will be slowest).
So I want to create one request such that the backend runs the queries in parallel and writes each result as it is ready and for the frontend to handle each result as it arrives. At the HTTP level I believe it can be just one body with serveral json, or maybe multipart response.
Is there an angularjs extension that handles the frontend side of things? Optimally something that works well with whatever can be done in the Java backend (didn't start investigating my options there)
I have another suggestion to solve your problem, but I am not sure you would be able to implement such a thing as from you question it is not very clear what you can or cannot do.
You could implement WebSockets and the server would be able to notify the front-end about the data being fetched or it could send the data via WebSockets right away.
In the first example, you would send a request to the server to fetch all the data for your dashboard. Once a piece of data is available, you could make a request for that particular piece and given that the data was fetched couple of seconds ago, it could be cached on the server and the response would be fast.
The second approach seems a more reasonable one. You would make an HTTP/WebSocket request to the server and wait for the data to arrive over WebSocket.
I believe this would be the most robust an efficient way to implement what you are asking for.
https://github.com/dfltr/jQuery-MXHR
This plugin allows to parse a response that contains several parts (multipart) by having a callback to parse each part. This can be used in all our frontends to support responses for multiple data (widgets) in one requests. The server side will receive one request and use servlet 3 async support (or whatever exists in other languages) to ‘park’ it, sending multiple queries, writing each response to the request as each query returns (and with the right multipart boundary).
Another example can be found here: https://github.com/anentropic/stream.
While both of these may not be compatible with angularjs, the code does not seem complex to port there.
Please presume that I do not know anything about any of the things I will be mentioning because I really do not.
Most OpenData sites have the possibility of exporting the presented file either in for example .csv or .json formats (Example). They also always have an API tab (Example API).
I presume using the API would mean that if the data is updated you would receive the change whereas exporting it as .csv would mean the content will not be changed anymore.
My questions is: how does one use this API code to display the same table one would get when exporting a .csv file.
Would you use a database to extract this information? What kind of database and how do you link the API to the database?
I presume using the API would mean that if the data is updated you
would receive the change whereas exporting it as .csv would mean the
content will not be changed anymore.
You are correct in the sense that, if you download the csv to your computer, that csv file won't be updated any more.
An API is something you would call - in this case, you can call the API, saying "Hey, do you have the latest data on xxx?", and you will be given back the latest information about what you have asked. This does not mean though, that this site will notify you when there's a new update - you will have to keep calling the API (every hour, every day etc) to see if there are any changes.
My questions is: how does one use this API code to display the same
table one would get when exporting a .csv file.
You would:
Call the API from a server code, or a cloud service
Let the server code or cloud service decipher (or "Parse") the response
Use the deciphered response to create a table made out of HTML, or to place it into a database
Would you use a database to extract this information? What kind of
database and how do you link the API to the database?
You wouldn't necessarily need a database to extract information, although a database would be nice to place the final data inside.
You would first need some sort of way to "call the REST API". There are many ways to do this - using Shell Script, using Python, using Excel VBA etc.
I understand this is hard to visualize, so here is an example of step 1, where you can retrieve information.
Try placing in the below URL (taken from the site you showed us) in your address bar of your Chrome browser, and hit enter
http://opendata.brussels.be/api/records/1.0/search/?dataset=associations-clubs-sportifs
See how it gives back a lot of text with many brackets and commas? You've basically asked the site to give you some data, and this is the response they gave back (different browsers work differently - IE asks you to download the response as a .json file). You've basically called an API.
To see this data more cleanly, open your developer tools of your Chrome browser, and enter the following JavaScript code
var url = 'http://opendata.brussels.be/api/records/1.0/search/?dataset=associations-clubs-sportifs';
var xhr = new XMLHttpRequest();
xhr.open('GET', url);
xhr.setRequestHeader('X-Requested-With', 'XMLHttpRequest');
xhr.onload = function() {
if (xhr.status === 200) {
// success
console.log(JSON.parse(xhr.responseText));
} else {
// error
console.log(JSON.parse(xhr.responseText));
}
};
xhr.send();
When you hit enter, a response will come back, stating "Object". If you click through the arrows, you can see this is a cleaner version of the data we just saw - more human readable.
In this case, I used JavaScript to retrieve the data, but you can use whatever code you want. You could proceed to use JavaScript to decipher the data, manipulate it, and push it into a database.
kintone is an online cloud database where you can customize it to run JavaScript codes, and have it store the data in their database, so you'll have the data stored online like in the below image. This is just one example of a database you can use.
There are other cloud services which allow you to connect API end points of different services with each other, like IFTTT and Zapier, but I'm not sure if they connect with open data.
The page you linked to shows that the API returns values as a JSON object. To access the data you can just send an appropriate http request and the response will be the requested data as a JSON. You can send requests like that over your browser if you want to.
Most languages allow JSON objects to be manipulated pro grammatically if you need to do work on the data.
Restful APIs publish model is "request and publish". Wen you request data via an API endpoint, you would receive response strings in JSON objects, CSV tables or XML.
The publisher, in this case Opendata.brussel.be would update their database on regular basis and publish the results via an API endpoint.
If you want to download the table as a relational data table in a CSV file, you'd need to parse the JSON objects into relational tables. This can be tricky since each JSON response string can vary in their paths.
There're several ways to do it. You can either write scripts to flatten the JSON objects or use a tool to parse and flatten the objects for you.
I use a tool called Acho to turn API endpoints into CSV files. It would parse almost all API endpoints through the parameters and even configure for multiple requests, such as iterative and recursive requests.
Acho API parser
I want to make a database query from frontend (Angular) to backend. But I need to send lots of parameters for that.
As far as I understand, if we are not making any database changes, it is better to use GET as it uses cached entries. POST should be used used if we need to make changes on server/DB.
But if I want to send many parameters (some are serialized objects) and make no server side changes, will it be alright to use POST request in that case and embed all parameters in the POST body instead of sending a huge URL encoded GET request?
To first clear this up: responses to POST requests can be cached, as long as the origin server returns the proper caching response headers. However, browsers and proxy servers generally don't apply caching to POST requests.
That being said, with the proper encoding you can store a lot of information in the ~ 2 KB of a query string, so GET should be the way to go.
If you're certain you'll go beyond the limits of a GET request and you'll need to go the POST way while remaining cacheable, you could introduce a kind of "nonce", for example using a hash of the search parameters:
Client does a POST to /search, with the search parameters.
Server stores the parameters somewhere, for example in a database.
Server generates a nonce from the parameters, for example by hashing the search parameters, or the row ID.
Server redirects the client to the result page for that nonce: /search/123abc.
Client requests the /search/123abc search results page.
Server performs the search based on the nonce, and returns a response which is cacheable.
This will introduce one additional HTTP roundtrip, but enable caching cross-browser and through proxy servers.
I think you should use post in this situation which is more manageable and looks clean. For more benefit of post follow these links:
Link 1
Link 2