i have a scenario where i have to load data from SQL server when i start running a web service. Later i have to use this data for my application, instead of accessing it every time from Database. In addition to this this data should be refreshed every one hour without affecting the website operation on the back end.If any of you has came across such scenario please let me know the solution. By the way i am using asp.net web services, SQL server database, and DNN for my front end.Thanks in advance.
In Global Asax,Application start event you can load all your data in the Dataset.
And by using Sql Cache dependency, You can refresh the data for each hour.But loading the
Entire data is not advisable.By making so you memory will be full.There will
be Performance degrade.
http://www.codeproject.com/Articles/14976/ASP-NET-Caching-Dependencies
Pre-loading all of your data is not a good practice because the database loses its purpose then. It’s probably ok for some data that is very rarely updated but needed very frequently but most definitely not for all the data you have in database.
As for the loading of data you can use app start event as others have already suggested.
Regarding caching – use Application object to make this data available to all parts of application and add a proprety to it that will keep the time of the last update. Then just create separate service that will check the last update time every X minutes and refresh the data when the time comes.
Related
I'm trying to figure out how to create an offline / online approch to use within a huge application.
Right now, each part of the application has its own model and datalayer, who directly read / write data from / to SQL. My boss is asking me to create a kind of buffer that, in case of connectivity failure, might be used to store data until the connection to SQL return active.
What I'm trying to create is something like this: move all datalayers into a servicestack service. Each "GET" method should query the database and store the result into a cache to be reused once the connection to SQL is not available. Each "POST" and "PUT" method must execute their actions or store the request into a cache if the connection fail. this cache must be cleared once the connection to SQL is restored.
How can I achieve this? Mine is a WPF application running on Windows 10.
Best regards
Enrico
Maintaining caches on the server is not going to help create an offline Application given the client wouldn't have access to the server in order to retrieve those caches. What you'd need instead is to maintain state on the client so in the event that network access is lost the client is loading from its own local caches.
Architecturally this is easiest achieved with a Web App using a Single Page App framework like Vue (+ Vuex) or React (+ Redux or MobX). The ServiceStack TechStacks and Gistlyn Apps are good (well documented) examples of this where they store client state in a Vuex store (for TechStacks created in Vue) or Redux Store (for Gistlyn created in React), or the Old TechStacks (created with AngularJS).
For good examples of this checkout Gistlyn's snapshots feature where the entire client state can be restored from a single serialized JSON object or approach used the Real Time Network Traveler example where an initial client state and delta's can be serialized across the network to enable real-time remote control of multiple connected clients.
They weren't developed with offline in mind, but their architecture naturally leads to being offline capable, courtesy of each page being first loaded from its local store then it fires off a Request to update its local cache which thanks to the reactivity of JS SPA fx's, the page is automatically updated with the latest version of the server.
Messaging APIs
HTTP has synchronous tight coupling which isn't ideal for offline communication, what you want instead is to design your write APIs so they're One Way/Asynchronous so you can implement a message queue on the client which queues up Request DTOs and sends them reliably to the server by resending them (using an exponential backoff) until the succeed without error. Then for cases where the client needs to be notified that their request has been processed they can either be done via Server Events or via the client long-polling the server checking to see if their request has been processed.
I am creating a webpage to show the listing from my database table,
suppose my database table is updating in every 10 second from my php cron , then i want to sync my backbone model/collection whenever my database table get update from my php cron.
Example:- i have table stock_exchange in which i am storing the stocks rate for various companies, and my cronjob update the stocks rate in every 10 second, To show this on UI i am creating the backbone application but my problem is that whenever my table stock_exchange updated i want to sync my backbone model/collection.
Please help,
Thanks in advance
You can either poll the server by calling stockModel.fetch() every 10 seconds in the browser (via setInterval perhaps), or you can use something like web sockets (via socket.io perhaps) to allow the server to push the latest data to the browser, which you can then do stockModel.set(dataFromServer);. Try something and post some code as stack overflow is intended for specific problems not tutorials.
Is this possible to prevent database connection while I requests concrete action in Zend Framework 2?
As Sam stated in a comment; if you do not wish to establish a connection to the database, then simply don't. If you do not have any code within your controller action that uses the database, then there won't be a database connection (provided that you are not doing funky stuff in a bootstrap method or similar).
If you are building a database connection in the standard way, then the actual connection will be lazily loaded, meaning that there won't be an actual connection before you try to use it.
In your comment you state that you believe the problem is caused by many database connections. I just want to clarify that there will only be one database connection per request (provided that you make use of the database).
If you have no code that calls a database, then your web server won't actually make a connection to your database. Either way, if you have 150+ images per page, then that would be a bigger concern and is probably the root cause for slow page loads. Perhaps consider pagination or if you do not display the pictures in their full sizes, then avoid scaling them in HTML as you would then be sending lots of unnecessary data from your web server to your visitors. You could resize the pictures in PHP when they are added, for instance. After that, you could even consider using a Content Delivery Network (CDN), but that is a different discussion...
I am using a combination of JSF,Servlets and Beans for a web application.The thing is I hit the database for some data and this data is populated on the user page in a chart manner using JFree chart.But the problem is,I dont want to hit the database each time,one query is enough to get all the data.So I want a logic which hits the database only once and gathers the data perhaps in a bean and then populates it according to the user request.Can this be done in my same application,should I be using Java Script to do this.I have the logic to populate the data but I dont have any idea where to put the Data Access Object method so that it executes only once.
You can use Spring cache-abstraction, and cache the data using EHcache
I am building a website (probably in Wordpress) which takes data from a number of different sources for display on various pages.
The sources:
A Twitter feed
A Flickr feed
A database on a remote server
A local database
From each source I will mainly retrieve
A short string, e.g. for Twitter, the Tweet, and from the local database the title of a blog page.
An associated image, if one exists
A link identifying the content at its source
My question is:
What is the best way to a) store the data and b) retrieve the data
My thinking is:
i) Write a script that is run every 2 or so minutes on a cron job
ii) the script retrieves data from all sources and stores it in the local database
iii) application code can then retrieve all data from the one source, the local database
This should make application code easier to manage - we only ever draw data from one source in application code - and that's the main appeal. But is it overkill for a relatively small site?
I would recommend putting the twitter feed and flickr feed in JavaScript. Both flickr and twitter have REST APIs. By putting it on the client you free up resources on your server, create less complexity, your users won't be waiting around for your server to fetch the data, and you can let twitter and flickr cache the data for you.
This assumes you know JavaScript. Once you get past JavaScript quirks, it's not a bad language. Give Jquery a try. JQuery Twitter plugin Flickery JQuery plugin. There are others, that's just the first results from Google.
As for your data on the local server and remote server, that will depend more on the data that is being fetched. I would go with whatever you can develop the fastest and gives acceptable results. If that means making a REST call from server to sever, then go for it. IF the remote server is slow to respond, I would go the AJAX REST API method.
And for the local database, you are going to have to write server side code for that, so I would do that inside the Wordpress "framework".
Hope that helps.