I am working on a grid that users may load xml file and do editing via grid panel. It works well with around 200 rows. However some users may want to load 50000 records at a time.
When I try to load a file with 10k records, firefox crashes and chrome waits too much. After some debugging I realized that rendering the data to panel requires too much time. for every new row I need to refresh the grid which requires too much time again.
One important point is, the xml file never comes to the serverside, we read it with extjs and load to the grid.
Is is possible to load whole data to store but request a certain amount of data to render the panel view?
I tried inifinite grid but cannot make it work since my data source is a string. What do you offer to manage 50k data.
I've had a similar issue with ExtJS in the past while trying to load very large grids. Let me just say at the outset that it's certainly not their fault - 50,000+ records is just way too much data. To get around this problem I resorted to using paging.
Have a look at their documentation for a paging toolbar.
Here is an example.
My only other suggestion would be to use an infinite grid, but write a file conversion wrapper over your XML data to convert it to JSON.
Related
I am getting 5000 list of user and their details from server and am trying to show those records . Records are displayed but am seeing continues loading.. and after loading am not able to do anything. Is there any solution to load 5000 records without loading and crashing?
The problem with loading 5000 records isn't (only) that you're receiving a lot of data, but that Angular has to process all that in the digest cycle. I'll bet that if you do a profiling in the browser, you'll find most of the CPU time is spend in the digest loop.
To avoid that you have to only allow a limited number of DOM elements on the page at a time. As suggested, one way of doing this is using paging.
Another way, where you still load your 5000 records, is to use a concept called virtual repeat/scroll, where you have a directive that only shows the elements that can fit in your viewport (i.e. on the users screen) and then whenever the user scrolls, the directive adds more elements below, while removing elements that goes out of view. This can be done seamless and can have a huge impact on performance.
Virtual repeat/scroll can be a difficult task to create on your own, but fortunately there are several different implementations available. I have experience with Angular Virtual Scroll ngRepeat directive but there are others out there that work just as well.
It is not advised to load 5000 records at once and try to display the same in the browser. You need to do use paging. You cannot load some heavy data onto browser. It might crash or does not respond. Use paging to bring only 10/20/30 records at a time and provide paging buttons , so user can click and navigate to the desired page.
An article on how you can do paging: http://www.c-sharpcorner.com/article/server-side-pagination-using-angularjs-web-api-and-sql-server/
--If you are using WebAPI
I need to show some images that queried form database and placed into inputstream.My framework is JSF and I know that by using servlet, I can show them. But the problem is that there is many images in my page that placed into database, now if I want to select each image from database and show in my xhtml page, a lot of queries needed. In one managedbean, all of images placed into List of inputstreams, and I want to show each element as an image in page. In fact my requirement is to read image from inputstream and show in xhtml page.Can any body guide me?
If you're using Richfaces, you can use <ui:repeat> to iterate your list of images and use <a4j:mediaoutput> to show your images on your xhtml, example, also see How to use a4j:mediaOutput correctly
and another example
now if I want to select each image from database and show in my xhtml page, a lot of queries needed
How exactly does that form a problem? Have you measured the performance? Is the bottleneck really in "a lot of queries"? I really don't understand why that would form a bottleneck. It should be blazing fast with a properly designed datamodel, a self-respected SQL database is designed for exactly this purpose.
Isn't your bottleneck actually the step of making a DB connection and that you're doing that on every single query because you aren't using a connection pool? If so, then yes, it would be understandable that it would perform very slow. Making a DB connection can be as slow as 100~500ms. That's exactly why connection pools were invented a long time ago. Connections would then only be initialized and cleaned during "idle time" and shared/reused on a threadsafe manner and thus getting a connection from it should be no more than 10ms or something.
If you fix your data layer to utilize a decent connection pool, then you can keep using your servlet which is already the right tool for the particular job.
I've a radiantQ gantt control and 20.000 data recived from wcf service to display. When less data to display it works like a charm. But i need to diplay 20.000 records and all browsers crash-freeze. Is there any way to solve this problem? Increasing isolated-storage may works?
thanks.
#halil ibrahim,
Please contact RadiantQ Technical Support and we will provide you hints on how to optimize gantt usage with large amounts of data.
Does application crash in rendering mode (when you display data on UI)? - You can use virtualization mode. You can try create one more thread (task or use background worker) and display parts of data to UI in "stack" mode. Do not load all data on first step. It should work. I tried do it: load of 1000+ data items from service and display it on UI.
Does application crash in process of receiving data from service (when you call a service method)? - You should configure your service. Increase max receive message size, etc. It depend on service, which you use.
Here's the story: I've been working on putting together an app in VS10 using Windows Forms in C++. This is my first experience with winforms, although I have a fair amount of experience with C++ and have worked with MFC a bit. The program needs to aquire data input from a hardware measurement system and the backend for that is mostly worked out. What I've been a bit stuck on is how to easily work with displaying this data in forms with the DataGridView and Chart objects. I'd also like to be able to save various sets of data and the programs current settings to disk that can be easily recalled and displayed in the program at a later point.
The solution I've been attempting so far has been to create a DataSet object for a certain program instance which holds some DataTables for the data and current settings. This is easily saved and read to XML files using the DataSet methods, which is nice. The tricky part for me using this method has been trying to sync up the program's display data and settings in the DataGridViews and Charts. It seems like these controls are really meant to be designed through the GUI. Maybe I'm just a noob, but I couldn't figure out how to get the DataSet I designed to link with my Charts using the "Add Project data source" dialog, so instead I set up a bindingSource and then proceeded to set Chart settings manually in the code. That's been getting hairy pretty quickly.
What would you have done to implement this, and how would you advise proceeding? I'm sure there has to be a much simpler way of doing this.
I've used Microsoft Chart Controls to display data from hardware measurement systems before with good results.
We store our data separately in arrays and when we want to display something, just call the AddXY function on the items to add a point. Configuring the graph in the first place is more easily done in designer though. Have a look at the samples for more info.
I have a Silverlight application that has a DataGrid and a DataPager. The data source for these controls comes from a database. I am accessing this database through RIA Services.
When I try to load all of the records, I receive an error that says:
"Load operation failed for query 'GetData'. The server did not provide a meaningful reply; this might be caused by a contract mismatch, a premature session shutdown or an internal server error."
By gradually restricting the size of the result set on the server side, I have come to the conclusion that I am getting this error because my data set is too large. My question is, how do I elegantly load large data sets into a DataGrid? I am open to approaches outside of RIA Services.
Thank you!
First off, if you have the means and aren't required to write this code yourself, consider buying a UI component that solves you problem (of find an open source solution). For these types of tasks, there's a good chance that someone else has put a lot of effort into solving problems like this one. For reference, there's a teleric grid control for Silverlight with some demos.
If you can't buy a component, here's some approaches I've seen:
Set up a paging system where
the data for the current page is
loaded, and new data isn't loaded
until the pages are switched. You
could probably cache previous results
to make this work more smoothly.
Load data when needed, so when the user scrolls down/sideways, data is loaded once cells are reached that haven't had data loaded.
One last idea that comes to mind is to gzip the data on the server before sending. If your bottleneck is transmission time, compression speed things up for the type of data you're working with.
You should take into consideration that you are possibly exceeding the command timeout on your data source. The default for LINQ to SQL for example is 30 seconds. If you wanted to increase this, one option is to go to the constructor and change it as follows:
public SomeDataClassesDataContext() :
base(global::System.Configuration.ConfigurationManager.ConnectionStrings["SomeConnectionString"].ConnectionString, mappingSource)
{
this.CommandTimeout = 1200;
OnCreated();
}