I'd like to set up a coldfusion page that will pull the status updates from my own facebook account and twitter accounts and put them in a SQL database along with their timestamps. Whenever I run this page it should only grab information after the most recent time stamp it already has within the database.
I'm hoping this won't be too bad because all I'm interested in is just status updates and their time stamps. Eventually I'd like to pull other things like images and such, but for a first test just status updates is fine. Does anyone have sample code and/or pointers that could assist me in this endeavor?
I'd like it if any information relates to the current version of the apis (twitter with oAuth and facebook open graph) if they are necessary. Some solutions I've seen involve the creation of a twitter application and facebook application to interact with the APIs; is that necessary if all I want to do is access a subset of my own account information? Thanks in advance!
I would read the max(insertDate) from the database and if the API allows you, only request updates since that date. Then insert those updates. The next time you run you'll just need to get the max() of the last bunch of updates before calling for the next bunch.
You could run it every 5 minutes using a ColdFusion scheduled task.
How you communicate with the API is usually using <cfhttp />. One thing I always do is log every request and response, either in a text file, or in a database. That's can be invaluable when troubleshooting.
Hope that helps.
Use the cffeed tag to pull RSS feeds from Twitter and Facebook. Retain the date of the last feed scan somewhere (application variable or database) and loop over the feed entries. Any entry older than last scan is ignored, everything else gets committed. Make sure to wrap cffeed in a try/catch, as it will throw errors if the service is down (ahem, twitter) As mentioned in other answers, set it up as a scheduled task.
<cffeed action="read" properties="feedMetadata" query="feedQuery"
source="http://search.twitter.com/search.atom?q=+from:mytwitteraccount" />
Different approach than what you're suggesting, but it worked for us. We had two live events, where we asked people to post to a bespoke Facebook fan page, or to Twitter with a hashtag we endorsed for the event in realtime. Then we just fetched and parsed the RSS feeds of the FB page, and the Twitter search results, extracting what was new, on a short interval... I think it was approximately every three minutes. CFFEED was a little error-prone and wonky, just doing a CFHTTP get of the RSS feeds, and then processing the CFHTTP.filecontent struct item as XML worked fine
.LAG
Related
I am building an sample application that lets user store comments.
I've created the registration and login process. When the user registers, his details are stored in a MySQL database and a token is returned to the browser. Now he can access the Profile page.
When an existing user logs in he is redirected to the profile page. The profile page is accessible only when a user registers or logs in.
After logging in, I want to show all his comments if he has already added them.
My frontend is in Angular and backend use Laravel. For authentication I use Satellizer.
I want to know, what is the best approach while playing with data, considering the fact that the user will add, edit his comments. Should I use localstorage and store data in a key value pair or should I create a json file which gets updated everytime the user adds a comment or makes a change.
I wanted to know what is the most efficient way to deal with data from server so that the application is fast even when it scales to a 10000 users and lot of data for each user.
Thanks
You should be updating it on the server when changes are made rather than only relying on localstorage. You can use localstorage to cache, but it should only be for immutable data, it shouldn't really be used for data that is going to change.
So in this case you'll be adding and updating new comments via your API (ideally a RESTful one!). Once you've made a change, you could store the comments locally and only update them when the user makes a new comment, however you'll quickly run into issues where the data is invalid on different clients. (i.e. if you update the comments on a different computer, the other computer won't be aware).
Alternatively, you could cache the comments and then simply ping the server to find out if new comments have been added. This could be using a HEAD request for example to check the last modified date on your comments resource.
You can store comments data locally on user browser, but you should properly manage it.
I don't how much load your server will have and if the time invested now worths it.
You can fetch comments and store them locally
User adds a comment, then you update locally and send a request to the server
You need to track the request response, if requests fail so notify user and remove comments from local.
if request was successful so you can continue on your way.
** facebook uses this "success first" approach
user does an action, and he see it happens instantly, in the background it could take few seconds, only if it fails they will notify you.
** look at their commenting process, when you comment, it appears instantly, no loading... but in the BG the load happens.
One feature on my site allows registered users to create calendars for their organization. We provide a dynamically-generated iCal feed for these calendars through a URL with query-string parameters. Anyone can subscribe to these feeds by entering the provided URL into Google Calendar, Outlook, iPhone, etc...
This has been working well enough for a few years, but we now have a problem with stale or deleted calendars. If a registered user significantly alters or deletes their account, the calendar will no longer exist and the feed is useless. We currently return a "404 - Not Found" error for those requests (recently changed from "400 - Bad Request").
My question is, other than returning the 404, is there any way to get subscribers to stop requesting a bad feed? This is a similar question, where the accepted answer suggests returning 404 or 410 and hoping the clients will see the error and manually remove the subscription.
That doesn't seem to be working so far. We get ~ 100k feed requests an hour and a full 30% of those are for deleted calendars.
Do Google, Apple, et al not give up when they repeatedly get a 404 for a feed? How have others handled this issue?
If this was just a problem with log pollution I wouldn't worry too much about it. However, since the feeds are dynamically generated, each request hits the backend db. The processing is trivial and doesn't appear to be affecting performance, but the situation can only get worse.
Apologies if this belongs on ServerFault. While the issue affects my servers, I believe the solution is programmatic.
I don't believe there is an easy answer - I think it's been asked before.
It's like having to deal with all the traffic when some hackers use your site for target practise on logins or xmlrpc or just looking for vulnerabilities. Or the spammers trying a scatter gun approach sending emails. Or when a web spider decides to excessively crawl your site. You have to size for all that non useful traffic.
You could possibly generate and keep up to date a list outside of the database of the bad ics URLs and have a script check and bounce the request before it gets near the database ?
Basically try to deal as efficiently as possible with the problem.
You could also in account deletion try adding a step that requests the user go to their calendar programs and delete the feed before continuing. However that might cause bad vibes and probably would not totally fix it anyway.
I have an internal corporate web application that has a page that hosts static content (things like announcements) and I decided to implement OutputCaching on this page to reduce the ammount of processing and db calls.
The only problem is that I need to also keep track of who has and hasn't viewed the content. Specifically I need to be able to audit who has seen a specific announcement (The announcement is determined by the GUID id passed to the MVC endpoint)
Is there any way to log who has accessed a page that has been output cached? Anything would be better than nothing, but if the log could be sent to an SQL database, that would be best solution for me.
I would normally log using an ActionFilterAttribute, but according to the stackoverflow article Working with the Output Cache and other Action Filters, that does not work well with the default output cache and instead you could try an alternative called DonutOutputCache.
I am building a web app that will use an auto-complete/suggestions for the end user as they type their information in. This will be specifically for entering Country, Province, City information.
Do a wild card search on the database on each keystroke:
SELECT CityName
FROM City
WHERE CityName LIKE '%#CityName%'
Return a list of all Cities to a given Province to the client and have the client do the matching:
SELECT CityName
FROM City
WHERE ProvinceID = #ProvinceID
These would be returned to the client as a JSON string via an ajax call to a web service. My thoughts are that javascript would be able to handle the list of 100+ entries via JSON faster than the database would be able to do a wildcard search, but I'd like the communities input.
In the past, I have used both techniques. If you are talking about 100 or so entries, and assuming each entry is very small, it will likely be faster to do the autocomplete filter on the client side. That will provide you with better response time (although probably negligible) and will reduce the load on your server.
Google actually does a live search while the user is typing, and it seems to be pretty responsive from the user's point of view. This is an example where the query must be executed server-side because the dataset is far too large to transfer to the client.
One thing you might do is wait until the user types two keystrokes before fetching the list from the server, thus narrowing down the results initially. Of course, that adds complexity - you would then need to refresh the list if the user changes either of the first two keystrokes.
We have implemented same functionality using ajax auto complete control we wait the user type three keystroke before fetching the list from server we have not done any coding at client side we just assigned web services method which return list to ajax control and its start working
In the end user's interest, it is always better to handle this client-side.
The Telerik Autocomplete controller allows for both ways.
Of course under load client-side autocomplete is likely to make the application crawl.
I'm trying to write an app on the SalesForce platform that can pull a list of contacts from a report and send them to a web service (say to send them an email or SMS)
The only way I can seem to find to do this is to add the report results to a newly created campaign, and then access that campaign. This seems like the long way around.
Every post I read online says you can't access the reports through Apex, however most or all of these posts were written before Version 20 of the API was released last month, which introduced a new report object. I can now programmatically access info about a report (Such as the date last run etc) but I still can't seem to find a way to access the result data contained in that report.
Does anyone know if there's a way to do that?
After much research into it, I've discovered the only way to do this at the moment is indeed to scrape the CSV document. I would guess that Conga etc are using exactly this method.
We've been doing this for a while now, and it works. The only caveats are:
Salesforce username / password /
security token has to be shared to
the app connecting. If the password
changes (and by default it is changed
every 30 days or so) the token also
changes and must be re-entered.
You have to know the host of the account, which can be difficult to
get right. For instance while most european accounts would use emea.salesforce.com to access CSV, our account uses na7 (North America 7) even though we're located in
ireland. I'm currently sending the page host to the app and parsing it
to calculate the correct subdomain to use, but I think there has to be a
better way to do this.
Salesforce really needs to sort this out by supplying an API call which allows custom report results to be exported on the fly and allowing us to use OAuth to connect to it. But of course, this is unlikely to happen.
In the SalesforceSpring 11 update, it seems you can obtain more informations about the Reports:
As stated in the API for Report and ReportType, you can access via Apex the fields used in the query by the Report, reading the field "columns", as well as the field used to represent the filters called "filter".
Iterating through this objects, should allow you to build a String representing the same query of the Report. After building that string you can make a dynamic query with a Database.query(..) call.
It seems to be a little messy, but should work.. (NOT TESTED YET!)
As header states, this works only with Custom Reports!
Just to clarify for fellow rookies who will find this, when the question was asked you could access your report data programatically, but you had to use some hacky, error prone methods.
This is all fixed, you can now access your reports via the API as of Winter '14.
Documentation here - http://www.salesforce.com/us/developer/docs/api_analytics/index.htm
Go to town on those custom dashboards etc. Cross posted from the Salesforce Stack Exchange - https://salesforce.stackexchange.com/questions/337/can-report-data-be-accessed-programatically/
But Conga (appextremes) do this in their QuickMerge product, where the user specifies the report Id, and the apex script on the page runs the report to extract the results for a mail merge operation.
the v20.0 API added metadata about the reports, but no way to actually run the report and obtain the results. If this is a standard report, or a report you've defined, you can work out the equivalent SOQL query for your report and run that, but if its an end user defined report, there's no way to do this.