Scalable database query - database

I have a youtube style site but it revolves around pictures.
On the homepage I want to show latest pictures that have been uploaded above the most popular pictures of all time.
Is it a good idea to do a database/cache query for every user when they hit the page in order to check what the latest images are and display them or should I do this another way to ensure the database isn't constantly flooded with requests for the latest posted pictures?
Maybe some sort of batch job?
Any ideas?

The most basic proactive thing you can do here is to cache the results of the DB query - either in your app code (less preferable) or in an existing piece of infrastructure integrate-able in your web "stack", for example something like Memcached:
http://memcached.org/
This has helped many a DB-backed site achieve some minimal level of scalability/performance.
Depending on your DB, you can also cache such queries as part of DB functionality itself, but it's better if you can intercept such things before they even get to the DB.

Related

BLOB database setup

I collected lots of data (images, pdf's,...) over some years. For my start-up, I like to be able to access this data through a web application as well as a website. All of the data has a relation with each other reaching from example images, contact cards, and bills,... so I would like to query the data by inserting tag filters. I am a very beginner in this stuff but would love to get a good basis and self-education, so I can maintain and manage my database. A connection with normal SQL data is also necessary. I assume the setup of a central database to connect later on web applications/website/mobile apps is the first thing to create....? If somebody can help me at least with some advice on how to start, I would be very happy.
I searched a lot, but as a newbie I get overwelmed by all kinds of software.
I would love to get some help on how to start in the right way.

Persisting and keeping mobile app data in snych with online backend

I am building a mobile app using AngularJS and PhoneGap. The app allows the user to access a large amount of data-items. These data-items come with the app in form of a number of .json files.
One use-case is that a user can favorite any of those data-items.
Currently, I store the (ids of) the items that have been favorited in localStorage. It works and it's great and very simple.
But now I would like create an online-backend for the app. By this I mean that the (ids of) the items that have been favorited should also be stored on a server somewhere in some form of database.
Now my question is:
How do I best do this?
How do I keep the localStorage data and online-backend data in synch?
In particular, the user might not have an internet connection at the time were he favorites a data-item. Additionally, if the user favorites x data-items in a row, I would need to make x update calls to the server db, which clearly isn't great.
So, how do people do it?
Does Angular have anything build-in for this?
Is there any plugin?
Any other framework?
This very much seems like a common problem that must have a well-known solution?
I think you've almost got the entire solution. All you need to do is periodically (on app start load the data from the service if available, otherwise use current local storage, then maybe with a timer and on app close update the data if connected) send the JSON out to a service (I generally prefer PHP, but Python, Java, Ruby, Perl, whatever floats your boat) that puts it in a database. If you're concerned with merging synchronization changes you'll need to use timestamps in the data in local storage and in the database to make the right call on what should be inserted vs what should be updated.
I don't think there's a one size fits all solution to the problem, though I imagine someone may have crafted a library that handles the different potential scenarios the configuration may be as complicated as just writing the logic yourself.

Database creation and query

So I have to created a recipe website and HTML-CSS is mainly my forte. I need a database to search through over a 100 recipes and mainly sort them,by author, apart from the other sorting orders. I don't want to use a CMS like Joomla. How do I start about?
Do I store the entire recipe(with a picture or two), into the database, or only a link to the recipe?
Secondly, the client would be updating the website as well, is there any way to simplify the process for the client who has absolutely no knowledge of adding into a database.
You're going to need to do some server-side scripting. If you don't want to use a CMS or framework, you (or someone else) will have to write the code for all of the site.
DB design pointers:
Store the recipe in the database, along with the author, etc.
Don't store the pictures in the db, even though it's easy enough to do. Better store than in a field in the db, called 'filename' or something which stores the path of the images on the server.
For the client - you will need to build a backend/admin page(s) with 'forms' for the client to upload (add), update and delete recipes and pictures.
You don't need save pictures into database. See database model of Prestashop(see only relative to images because are various tables), for example.
Regards and good luck!
You can add pictures into data bases as well. For that you can always reduce the size of the images before inserting into database.
For database, you can use php or javascript. Both provide easy way of accessing database.
Javascript even has inbuilt transaction commit and rollback feature.

An app to search a database

Im not a developer (not a proper one) but always up for an excuse for self-development.
I work in a support team for a reputable software vendor, and we currently use a helpdesk piece of software called iSupport.
- Its not a bad piece of kit, and im not sure if it has been set up badly (trying to find out) but the biggest problem I face (being on the front line as an analyst who uses it 8hrs a day) is its inability to easily search.
A new 'incident' will come in. Client will report some errors in a log, perhaps mention some other keywords when describing his symptoms.
Now I know, I have probably answered a similar problem before (but cant remember the solution) or even more likely, a fellow-analyst may have answered the same question before.
I would like to have the ability to have one search box (think Google) that searches through EVERY incident that has ever been created and return me ALL incidents that contain that keyword.
At the moment the search is very poor - You can take time to set up searches and specify which fields you want to search on, which values to filter by (perhaps by an analyst or category, etc) but this takes time and more often than not, it returns poor results and it would have been easier to try and track it down yourself manually.
All of the data sits in underlying SQL Server tables (have requested a subset of the data).
What im thinking, is creating a separate front-end that is just a basic search box and thats it. This app will point to all the relevant fields in the SQL tables and pull out the relevant records into a table. Once I have the ID for the incident, it is then a simple job to pull out that incident back in the iSupport front end.
I was thinking along the lines of Google Desktop style app (shortcut key brings up the search box).
Now thats as much thinking as ive done. Looking for some advice on where to go next.
I know for instance, that Google Desktop crawls and indexes all your physical files on your machine. Would I have to do something similar for a database as I imagine there maybe a large number of records/fields/tables to search through.
TBH, if it works, im not that fussed (to begin with) if it takes awhile to process the query, as long as it returns relevant results. But ideally id like it to be quick.
Ill leave it at that for now.
Where should I begin?
If .NET is your thing, then Lucene.NET will work well with SQL Server to give you that google search feeling.
The StackOverflow websites use it, you can listen to the SO Podcast where Jeff/Joel bitch about why SQL Server full-text search sucks so much.
I'd suggest this might be a good candidate for a web application - an asp.net / jsf website. This means that you can control it from one machine, but all your colleagues can make use of it without a deployment headache every time you add a new feature...
The incident database is mission critical (critical to your relationship with customers), so if you came to me with this request I would insist that you accessed the database through a user that had select permissions to the appropriate tables, and very little else. This from your point of view is a good thing too - let's you operate knowing you're not going to cock anything up...
The SSMS Tool Pack (an add-in to Management Studio) contains a feature to search Table, View or Database Data.
Have a look into the Full-Text Search functionality of SQL Server.
iSupport includes a full-text search function. Add a 'Global Search' widget to a dashboard you create.

Is it better to have 1 page or many for a website listing a small number of products?

I have a small website for a company that have 40-50 product. The site will be in french and English. Each product will have 2-3 different price (based on shipping)
So the standard approach, will be to do 50 (100 in both language) different web page done with an template engine, maybe pure HTML and css coding
The other solution, make 1 page with PHP, get all the product in database and populate that single page, that will be harder to program, but the page will create itself
So the question : what YOU will do, what is the approach you will take
Take in account that the php-database option is a lot harder to make it work first !, all the time worth to get only that small count of product. I have no doubt if there is 2500 products.
Thanks in advance
Based on the 4 great answer, i will go with a database, let make it the right way, event if it a little bit more complicated
two more question,
what will be the code for the modrewrite
what software (web or mac) to populate/edit the sqllite database
I would definitely make the PHP page. If you have to make changes to the product pages, then all you have to do it do it in one place (in the PHP). It can make maintenance far easier in the future.
I would definitely go the single PHP programming route, but I would do a URL Rewrite so that each page appears to be its own page. This will help for SEO, etc.
So the pages might look like this:
http://www.example.com/products/redcar/
http://www.example.com/products/bluecar/
http://www.example.com/products/greencar/
But they actually point to http://www.example.com/product.php?product=redcar etc
Definitely go with the database and script approach. Adding a row to a db is a lot simpler than creating and managing yet another html file. Even with just 50 products, imagine if you wanted to increase the price of your products by 1.5%: with a db, it's a single query process; with static html you're stuck editing each and every page.
Something else to consider is that with a db, you are not limited in the ways you present your data. So as well as single product pages, you can also generate product category summary pages, search result pages, etc.
If you will be involved with this project in the future, OR if the product line will expand and you want to serve the company most effectively, then you should use the database approach.
If you won't be involved in the future, AND you have no experience with PHP and a database, AND you are used to a template engine that makes all this a snap, then I agree that you have a difficult decision.
I certainly would use the database approach; among other things, it would allow the language variation to be handled very elegantly. But that is a reflection of my experience.

Resources